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FOREWORD 


By Michael R. Salamone 
Staff Officer 
Transportation Research Board 


Recent accidents involving aircraft overruns focused attention on improving airport run- 
way safety areas in the United States and elsewhere. ACRP Report 3: Analysis of Aircraft 
Overruns and Undershoots for Runway Safety Areas, the culmination of ACRP Project 04-01, 
examines historical data related to both overrun and undershoot occurrences. It will assist 
airport operators and airport planners in identifying conditions that may contribute to 
overruns and undershoots occurrences at airports. 


ACRP Report 3: Analysis of Aircraft Overruns and Undershoots for Runway Safety Areas 
covers four areas: (1) Research collected on accident/incident data from several notable 
sources; (2) inventory of the conditions relating to each; (3) assessment of risk in relation 
to the runway safety area; and (4) discussion on a set of alternatives to the traditional run- 
way safety area. 

Overruns and undershoots are factors in the design or improvement of runway safety 
areas (RSAs). The traditional approach to mitigate risk associated with accidents or inci- 
dents is to enlarge the runway safety area, but many airports do not have sufficient land 
to accommodate standard Federal Aviation Administration or International Civil Avia- 
tion Organization recommendations for RSAs. Airports that pursue this approach face 
extremely expensive and controversial land acquisition or wetlands filling projects to make 
sufficient land available. 

This report uses a probabilistic approach—a quantitative assessment—to analyze the 
RSA and begins a discussion on how alternatives to a standard 1,000-foot RSA may ade- 
quately mitigate risk. The report also assesses the factors that increase the risk of such acci- 
dents occurring, helps with understanding how these incidents may happen, and suggests 
that aircraft overrun and undershoot risks are related to specific operational factors. 

The report suggests that significant improvement to airport operations safety may be 
achieved by monitoring and managing these operational factors for both RSA planning and 
during actual aircraft operations, and it provides recommendations for collection and 
reporting of data in future accident and incident investigations and reporting to allow future 
improvements to these models. 
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SUMMARY 


Analysis of Aircraft Overruns and Undershoots 
for Runway Safety Areas 


The objective of this project was to develop an aircraft overrun and undershoot risk 
assessment approach, supported by scientific evidence and statistical theory, which provides 
step-by-step procedures and instructions for analysis of runway safety areas (RSA). Most 
aircraft accidents occur during the landing and takeoff phases of the flight and are likely to chal- 
lenge the existing RSA when the aircraft overruns or undershoots the paved area of runways. 

Currently, Federal Aviation Administration (FAA) standards require runways to include 
an RSA: a graded and clean area surrounding the runway that “should be capable, under 
normal (dry) conditions, of supporting airplanes without causing structural damage to air- 
planes or injury to their occupants” (FAA, 1989). The RSA improves the safety of airplanes 
that undershoot, overrun, or veer off the runway. The size of the RSA is dependent on the 
type and size of aircraft using each runway. 

Most aircraft accidents occur during the landing and take-off phases of the flight and are 
likely to challenge the existing RSA when the aircraft overruns or undershoots the paved area 
of runways. The risks of an aircraft overrunning or undershooting a runway depend on a 
number of factors related to the operation conditions, like the weather, the runway surface 
conditions, the distance required to land or takeoff, the presence of obstacles, the available 
runway distance, and the existing RSA dimensions, just to name a few. The possibility of 
human errors or aircraft system faults during the landing or takeoff phases of the flight also 
may contribute to the risks. 

Based on information gathered from overrun and undershoot accident and incident reports, 
risk models that consider relevant operational factors were developed to assess the likelihood 
and possible consequences for such accidents occurring on a runway subject to specific traffic 
and operation conditions. The approach uses historical flight data and the configuration of 
existing or planned RSAs to evaluate the risk for each operation and derive its probability dis- 
tribution that allows for quantifying the number of high risk operations at the airport. 

The major achievement of this research is to provide an innovative, rational and compre- 
hensive probabilistic approach to evaluate the level of risk for specific airport conditions that 
will allow the evaluation of alternatives when FAA recommended RSA configuration for an 
existing airport cannot be met. In addition, based on the existing level of risk, this approach 
will allow prioritizing financial resources to improve safety areas, achieving target levels of 
safety (TLS) and helping with safety management actions when high risk conditions arise. 

The results of the current study can be used by a broad range of civil aviation organiza- 
tions for risk assessment and cost-benefit studies of RSA improvements. These organizations 
may include Federal and State agencies, the International Civil Aviation Organization 
(ICAO), and international civil aviation authorities, as well as airport operators, airlines, 
civil aviation associations and institutions, universities, and consultants. 


CHAPTER 1 


Background 


introduction 


From 1995 to 2004, 71 percent of the world’s jet aircraft 
accidents occurred during landing and takeoff and accounted 
for 41 percent ofall onboard and third party fatalities (Boeing, 
2005). Landing overruns, landing undershoots, takeoff over- 
runs, and crashes after takeoff are the major types of accidents 
that occur during these phases of flight. Records show that 
while most accidents occur within the boundaries of the run- 
way strip, most fatalities occur near but off the airport area 
(Caves, 1996). 

Currently, Federal Aviation Administration (FAA) stand- 
ards require runways to include a runway safety area (RSA)— 
a graded and clean area surrounding the runway that “should 
be capable, under normal (dry) conditions, of supporting air- 
planes without causing structural damage to airplanes or injury 
to their occupants” (FAA, 1989). Its purpose is to improve 
the safety of airplanes that undershoot, overrun, or veer off the 
runway. 

The size of the RSA depends on the type and size of aircraft 
using the runway. RSA standard dimensions have increased 
over time. The predecessor to today’s standard extended only 
200 feet from the ends of the runway. Today, a standard RSA 
can be as large as 500 feet wide and extend 1,000 feet beyond 
each runway end. The standard dimensions have increased to 
address higher safety expectations of aviation users and 
accommodate current aircraft performance. 

However, applying the new standards to existing airports can 
be problematic. Many runways do not meet current standards 
because they were constructed to an earlier standard. The prob- 
lem is compounded by the fact that the airports are increasingly 
constrained by nearby land development and other natural fea- 
tures, or they face costly and controversial land acquisition, or 
aneed for unfeasible wetlands filling projects. 

The runway safety area standards are prescriptive and its 
rigid nature results in “averaged” degrees of protection being 
provided across broad ranges of risk levels, such that certain 


airports have much higher tolerance to risk than others. Ide- 
ally the risk associated with specific airport and operation 
factors should be modeled to assess the level of safety being 
provided by specific conditions of existing or planned 
RSA. Some intuitively important factors that would affect 
risk, such as various environmental and operational charac- 
teristics of the airport, are not considered yet. 

In current risk assessment methods, factors that determine 
safety cannot be analyzed independently; however, a rational, 
systematic identification of safety influencing factors and 
their interrelationships has never been conducted. This situ- 
ation impedes the assessment of effects of safety improve- 
ment opportunities and, consequently, risk management. 

Moreover, most airfield design rules are mainly determined 
by a set of airfield reference codes, which only take into ac- 
count the design aircraft approach speed and the aircraft 
dimensions (wingspan or tail height). The resulting protection 
is segregated in widely differing groups that do not necessar- 
ily reflect many of the actual risk exposure factors. 


Project Objectives 


The original objective of this project was to collect histor- 
ical information related to overrun and undershoot accidents 
and incidents to develop a comprehensive and organized 
database with editing and querying capabilities, containing 
critical parameters, including aircraft, airport, runway, oper- 
ation, and causal factor and consequence information that 
could assist the evaluation of runway safety areas. 

The research team extended the project objective to 
include the development of risk models for overrun and un- 
dershoot events. The primary function of the risk models is 
to support risk management actions for those events by in- 
creasing the size of the RSA, removing obstacles, construction 
of arrestor beds or perhaps, where that is not possible, by the 
introduction of procedural measures or limitations for oper- 
ations under high-risk conditions. 


Three sets of models were developed in this study—landing 
overruns, landing overshoots, and takeoff overruns. Each set 
is comprised of three parts: probability of occurrence, loca- 
tion, and consequences. The models can improve the under- 
standing of overrun and undershoot risks and help airport 
operators manage these risks. 

Based on the information described above, the goals for 
this research project were extended to include: 


1. Development of a comprehensive database for aircraft 
overrun and undershoot accidents and incidents; 

2. Determination of major factors affecting the risks of such 
accidents and incidents; 

3. Description of how these factors affect operations and 
associated risks, to improve understanding on how these 
events may occur; 

4, Development of risk models for probability, location, and 
consequences for each type of accident: landing overruns 
(LDOR); landing undershoots (LDUS); and takeoff over- 
runs (TOOR); 

5. Development of a practical approach to use these models 
for assessing risks on existing RSA under estimated oper- 
ation conditions; 

6. Development of a list of relevant factors that should be 
reported for aircraft overrun and undershoot accidents so 
that availability of quality data can be improved for future 
studies; and 

7. Development of prototype software to evaluate risks 
under specific operation conditions that may serve as the 
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basis for creating analysis software that can be used to 
assess risks of aircraft overruns and undershoots. 


Applied to any specific airport, the analysis approach for 
RSA risk assessment developed in this study will allow users 
to determine if the risk is relatively high or low and whether 
there is a need for risk management action. The safety bene- 
fits provided by possible mitigation measures (e.g., increased 
size of RSA) can be evaluated using the same approach. 

In addition, three innovative techniques were incorporated 
to improve the development of risk models. One major im- 
provement in the modeling of accident occurrence is the use 
of normal operations (i.e., nonaccident and nonincident) 
flight data. With normal operations data (NOD), the number 
of operations that experience the factor benignly, singly, and 
in combination can be calculated, so risk ratios can be gener- 
ated and the importance of risk factors quantified. 

The second improvement is the use of normalization tech- 
niques to convert information to a standard nominal airport. 
Using such normalization procedure allows comparing acci- 
dent and NOD data for different operation conditions, thus 
creating a larger pool of relevant information. 

Finally, the models developed were integrated in a rational 
probabilistic approach for risk assessment of RSA. Based on 
historical information for flight operations and weather con- 
ditions, and considering the configuration of the RSA and 
presence of obstacles located close to the runway, the proba- 
bility distribution for accidents involving severe consequences 
may be estimated. 


CHAPTER 2 


Research Approach 


The work for this study was planned and structured as 
follows: 


1. Conduct a literature review and a functional hazard analy- 
sis (FHA) to improve the research team’s understanding 
of factors causing or contributing to aircraft overrun and 
undershoot accidents, as well as to identify existing ap- 
proaches, procedures and sources of data to support the 
development of risk models. 

2. Collect historical accident and incident data from the 
sources identified and selected, and develop a compre- 
hensive database of relevant accidents and incidents that 
included the causal factors, contributing factors, and 
operation conditions. 

3. Collect historical NOD to support the development of risk 
models. 

4. Transform the data to enable comparisons, thus increasing 
the pool of available information to develop risk models. 

5. Develop three sets of risk models for LDOR, LDUS, and 
TOOR. Each set included: a frequency model for assessing 
the likelihood of the event, a location model to estimate 
the probability the aircraft wreckage is located beyond a 
given distance from the runway, and a consequence model 
that integrates the previous frequency and location model 
to evaluate the probability of severe consequences. 

6. Develop of a probabilistic approach for the analysis of 
RSAs that incorporates the models developed in this study. 

7. Incorporate the approach and models in prototype soft- 
ware to demonstrate the feasibility of the analysis approach 
developed. 


The research plan followed the diagram depicted in Figure 1. 
The research team conducted a literature review followed by an 
FHA to identify important parameters associated with overrun 
and undershoot events, and collected the necessary information 
to potentially use as independent variables in the risk models. 

After those parameters were identified, the research team 
screened the existing incident and accident databases to 


locate the events considered relevant for this study. For each 
event, available reports and docket documents were obtained 
and were analyzed in search for the relevant information in- 
cluded in the ACRP 4-01 database. 

Data was collected and a database was created to save this 
information in an organized manner. When possible, gaps 
observed for important parameters were obtained from 
sources other than accident investigation agencies to com- 
plement the missing information. A statistical summary of 
the database was developed and is presented in this report. 

Statistical tools and software were used to develop risk mod- 
els for frequency and location for each type of accident. These 
models incorporate historical flight and weather conditions 
to evaluate the level of risk exposure for a specific runway. A 
rational probabilistic approach was developed to integrate these 
models and to assess the probability of severe consequences for 
these accidents. Finally, these models were integrated in proto- 
type software to facilitate the analysis. 


Literature Review 


Risk assessments are utilized in many areas of aviation, 
from designing aircraft systems to establishing operational 
standards and air traffic control rules. However, there is little 
information available for assessing the risk of accidents 
occurring near and at airports. Previous relevant studies for 
airports can be broadly categorized into four areas: airport 
design, third-party risk, facility risk, and operational risk. 

To assess risk from an airport design standpoint, the U.K. 
Civil Aviation Authority (CAA) Safety Regulation Group 
conducted a study on aircraft overrun risk, which guides air- 
ports on overrun risk assessment and provides advice on how 
to reduce it (CAA 1998). 

Another study under this category is AEA Technology’s risk 
assessment of airfield design rules (Eddowes et al. 2001). In 
this study, the authors reviewed design standards such as run- 
way length and reference codes, the runway end safety area 
(RESA), separation distances between runways and taxiways, 
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Figure 1. Research plan. 


and obstacle limitation surfaces. It made concrete recommen- 
dations for amending the International Civil Aviation Orga- 
nization (ICAO) Annex 14 safety areas to achieve a specific 
target level of safety. 

In the United States, studies also have been carried out to 
set criteria for the design of airport safety areas, particularly in 
California. Garbell (1988) pioneered the accident-potential 
concept that led to the adoption of safety areas at a number 
of airports. A 1990 FAA study (David, 1990) compiled data 
regarding the location of commercial aircraft accidents rela- 
tive to the runway involved. The database was used to validate 
the RSA dimensions adopted by the FAA, and it is still effec- 
tive today. 

There are only a limited number of general methodologies 
and models for assessing an airport’s third party risk (Piers, 
1996). They are derived principally from studies commis- 
sioned by the Dutch and British governments and their 
results are broadly similar (Ale and Piers, 2000). 

A third family of studies seeks to assess the risk that aircraft 
operations pose to specific developments near airports. 
Examples of such studies include one for the U.S. Nuclear 
Regulatory Commission dealing with the safety of nuclear 
power plants, as well as a study for the Department of Energy 
for assessing the risk of an aircraft crash into its nuclear 
weapons and material storage facilities (Eisenhut, 1973; NRC, 
1981). A study on Salt Lake City International Airport inves- 
tigated the crash probability at a hospital, a school, and a 
shopping mall nearby (Kimura et al., 1995). 

The final group of risk assessment studies concentrates on 
flight operational safety and is not strictly considered airport 
risk assessment. However, certain elements of these studies are 
very relevant to airport risk analysis. For example, a study on 
navigational aids established risk ratios for mostly airport fac- 
tors that influence the risk of approach and landing accidents 
(Enders et al., 1996). A related piece of research by the Flight 


Assessment 


Safety Foundation measured accident risk based on, among 
other things, airport conditions (Khatwa and Helmreich, 
1998). The ICAO’s Collision Risk Model (CRM) calculates the 
collision probability of an operation with obstacles of known 
location and size during an Instrument Landing System (ILS) 
approach. The model is used as a decision-making tool for 
developing safe approach procedures and for airport planning 
(ICAO, 1980). 

One of the core reasons for oversimplification of accident 
frequency modeling is the lack of data on exposure to various 
risk factors in normal operations. Without NOD, crash rates 
related to the presence of risk factors cannot be established. 
Closing this gap in research is a major achievement of the 
work conducted by Loughborough University (Wong, 2007) 
and ACRP 4-01. 

Appendix A of this report provides information collected 
during the literature review on the procedures used and re- 
sources available to pilots during the landing and takeoff phases 
of the flight. Moreover, it describes how weather conditions, 
runway conditions, faults, and human errors can affect the 
operations and lead overruns and undershoots. 


Functional Hazard Analysis 


An FHA is a formal and systematic process for the identi- 
fication of hazards associated with an activity. The purpose of 
the FHA in the context of this study was to determine rele- 
vant causal factors of overrun and undershoot accidents and 
hazards to aircraft associated with airport operations (e.g., 
landing, takeoff roll, and associated fault sequences) and the 
physical design of airports. 

The risk analysis approach utilized in this study is based pri- 
marily on a review of operational experience, in particular, 
accident, incident, and normal operation data. The modeling 
approach adopted for the quantitative assessment of the risks 
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associated with runway operations was based on the evalua- 
tion of: 


The likelihood of the incident occurring; 

The location where the aircraft came to stop, in case of 
overruns, or its point of first impact, for undershoots; and 
The consequences of such an incident (injury and cost of 
damage). 


Overrun and undershoot incidents may be considered in 
terms of the deviation of the aircraft from its intended path. 
The definition of the deviation for each incident type is sum- 
marized as follows: 


For overrun incidents, the “longitudinal deviation” is de- 
scribed by the longitudinal distance traveled beyond the 
accelerate/stop distance available (for takeoff events), and 
beyond the landing distance available (for landing events). 
For undershoot incidents, the “longitudinal deviation” is 
described by the longitudinal distance the aircraft under- 
shoots the intended runway threshold. 

For both overrun and undershoot events, the “lateral 
deviation” is the lateral distance to the extended runway 
centerline. 


Examples of incident and accident causal factors include 
human error, as well as incorrect approach speed; deviation 
of approach height relative to desirable flight path; improper 
touchdown location; inappropriate runway distance avail- 
ability, aircraft system faults, improper weight, and aircraft 


configuration; low friction runway surface conditions (wet, 
icy, or contaminated); adverse weather conditions, particu- 
larly tail wind, cross wind, gusting wind, low visibility, and 
precipitation; and unfavorable runway slopes. Results for 
FHA of aircraft overruns and undershoots are provided in 
Appendix B. 


Database Development 


A single database that contains a significant number of rel- 
evant accidents and incidents on and near airports was cre- 
ated for this study. A second database comprising normal 
operations data also was developed for this study. The data 
were organized to facilitate the assessment of each accident 
type in a coherent manner, rather than based on multiple 
databases with different inclusion criteria. 

Before data were collected, some criteria were established 
for filtering out events available in the database sources that 
would not be relevant for ACRP 4-01 model development. It 
is important to describe the criteria used and the reasons for 
applying them. 


Filter Applied to the Data 


Some filtering criteria were used on the data so that events 
were comparable, as well as to ensure the models developed 
would represent the objectives of this study. The first filter 
was an attempt to use information from only specific regions 
of the world having accident rates comparable to the U.S. 
rate. Figure 2 depicts accident rates by region of the airline 


Western-built transport hull loss accidents, by airline domicile, 


1994 through 2003* 
Europe C.LS.1 
tha-08 
United States JA 8 
jultod Biase Non JAA - 1.2 0.5 
0.4 Middle East 
3.1 Asia (Excluding 
1.7 ® China) 
Africa 
13.3 
Latin America 
and Caribbean 
2.4 
Oceania 
0.0 
World 
1.0 


‘nsufficient fleet experience to generate reliable rate. 
* 2003 Preliminary Information 


Rates per million departures 


Source: U.S. Department of Transportation, Federal Aviation Administration. 


Figure 2. Accident rates by region of the world. 


domicile. It was assumed that the information from North 
America (United States and Canada), Western Europe (Joint 
Aviation Authorities [JAA] countries), Oceania, and a few 
selected countries in Asia would be relevant to this study and 
included in the database. 

Using this filter criterion, the main sources of data included 
the following: 


e FAA Accident/Incident Data System (AIDS); 

e FAA/National Aeronautics and Space Administration 
(NASA) Aviation Safety Reporting System (ASRS); 

e NTSB Accident Database & Synopses; 

e Transportation Safety Board of Canada; 

e ICAO Accident/Incident Data Reporting (ADREP) system; 

e Australian Transport Safety Bureau (ATSB); 

e France Bureau d’Enquétes et d’Analyses pour la Sécurité de 
PAviation Civile (BEA) ; 

e UK Air Accidents Investigation Branch (AAIB); 

e New Zealand Transport Accident Investigation Commis- 
sion (TAIC); 

e Air Accident Investigation Bureau of Singapore; 

e Ireland Air Accident Investigation Unit (AAIU); and 
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e Spain Comisién de Investigacién de Accidentes e Inci- 
dentes de Aviacion Civil (CIAIAC). 


A more detailed description of each data source is available 
in Appendix C of this report. In addition, the filtering crite- 
ria and justification described in Table 1 were applied to the 
ACRP 4-01 database. 


Data Limitations 


There are many quantitative and qualitative limitations to 
reliable accident and incident data, and these limitations in- 
variably constrain the depth, breadth, and quality of airport 
risk assessments (Piers et al., 1993; DfT, 1997; Roelen et al., 
2000). This study is no exception. The scope and detail of the 
analysis are restricted by the availability and quality of the 
data extracted from available sources. Major data limitations 
found during the collection process are outlined in the fol- 
lowing material. 


Missing Data. Accident investigation records and incident 
reports consist of a number of standard forms and reports. 


Table 1. Filtering criteria for accidents and incidents. 


Justification 
Study is concerned with fixed wing aircraft 
accidents and incidents only. 


Filter #__ Description 

1 Remove non-fixed wing aircraft 
entries. 

2 Remove entries for airplanes 


with certified max gross weight 
< 6,000 Ibs (<12500 Ibs if Part 
91). 

3 Remove entries with unwanted 
Federal Aviation Regulation 


(FAR) parts. Kept Part 121, 125, 


129, 135 and selected Part 91 
operations. 


4 Remove occurrences for 
unwanted phases of flight. 


Cut off criteria for lighter aircraft utilized 
to develop model for overruns on unpaved 
areas, 


Some FAR parts have significantly 
different safety regulations (e.g., pilot 
qualifications). The following cases were 
removed: 

Part 91F: Special Flt Ops. 

Part 103: Ultralight 

Part 105: Parachute Jumping 

Part 133: Rotorcraft Ext. Load 

Part 137: Agricultural 

Part 141: Pilot Schools 

Armed Forces 

Keep only undershoots and runway 
excursions beyond the departure end of the 
runway. We are also keeping veer-off 
occurrences that were available in the 
Loughborough University database, but 
these will not be utilized for developing the 
tisk models. 


3 Remove all single engine aircraft 


and all piston engine aircraft 
entries. 


Piston engine aircraft are now used less 
frequently in civil aviation and therefore 
have been removed, to increase the validity 
of the modeling. Moreover, single and 
piston engine aircraft behave differently in 
accidents due to the lower energy levels 
involved and the fact that the major focus 


of this study is air carrier aircraft. 


6 Remove all accidents and 
incidents when the point of first 
impact and the wreckage final 
location is beyond 2000ft from 
threshold. 


It would be unfeasible to have an RSA with 
more than 2000ft beyond the threshold, the 
gain in safety may not be significant and a 
previous FAA study used this criterion 
(David 1990). 
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Even within these standard areas of interest, it is extremely rare 
that every field is complete. The NTSB docket files of minor 
accidents, when available, frequently contain less than a dozen 
pages of forms accompanied by only a brief synopsis of the 
occurrence. Information for incidents is very poor. As an 
example, prior to 1995, the narratives for AIDS reports were 
limited to 115 characters; reports since 1995 contain a more 
complete narrative prepared by the investigating inspector. 


The accident wreckage site often is given only a very crude 
description without supporting maps or diagrams. Only a 
small proportion of data fields are systematically recorded for 
every accident. The amount of missing fields in the database 
is high, restricting the number of parameters that could be 
analyzed with confidence. In many cases the report descrip- 
tions used were coupled with the runway satellite picture 
obtained from Google Earth to determine the approximate 
location of the accident and of the wreckage. 


The reports contain mostly information the accident inves- 
tigators deemed relevant to an accident’s occurrence. Outside 
of this judgment, few potential risk factors and measurements 
are included. This was a major obstacle to developing a data- 
base that consistently and systematically records a compre- 
hensive set of risk exposure parameters so they could be 
included in the models. The data available for analysis and 
model-building ultimately depended on the agency accident 
investigation mentality and policies. There are no alternative 
sources of data, and this issue is particularly critical concerning 
unconventional or latent risk factors beyond the well estab- 
lished sources of risk. Parameters such as weight and runway 
criticality that would require additional calculation often are 
impossible to compute because of unavailable data. 


When considering aircraft overruns, there are flights that 
used more than the nominal runway distance required (take- 
off or landing) to complete the operation but without 
departing the runway due to excess runway length available. 
These cases usually are considered as normal operations and 
will never feature in accident records. 


Although these cases could provide additional and valu- 
able data to model incident location distribution, obtaining 
normal operations data on actual runway distance used 
proved difficult to obtain despite extensive efforts from the 
research team. 


Finally, the presence of excess runway may alter pilot 
behavior such that more runway distance is used than other- 
wise. In a number of occurrences, the pilot did not apply 
braking to stop the aircraft as soon as possible because the 
pilot elected to take a specific taxiway exit or was hurried by 
the traffic control to quickly leave the runway. 


However, it should be noted that these data limitations are 
not unique to the current research but are inherent to risk 
assessment studies that use historical accident data (Piers, 
1994; ETSC, 1999). 


Poor Data Quality. Previous studies using data from 
accident reports and docket files have reported on the poor 
quality of data available (Hagy and Marthinsen, 1987). Erro- 
neous or conflicting information within the same docket is 
not uncommon. Some cases were identified where the pro- 
vided wreckage location diagram does not match the text 
description given. Confusing and inconsistent use of terms 
and nomenclature adds to the challenges of extracting precise 
data points. When faced with conflicting data, the research 
team applied judgment to obtain a final figure according to 
the best information available. 


Measurement Difficulties. The measurement of certain 
parameters suffers from inherent ambiguity in the aviation 
industry. A prime example is runway condition. There sim- 
ply has not been an agreed industry standard on reporting 
runway conditions and determining its relationship with 
runway friction and aircraft braking performance (DeGroh 
2006; FAA, 2006b). The current industry approach is to 
measure and report runway friction periodically using stand- 
ard equipment and wet surface conditions. However, it also 
is common practice to rely on pilots’ subjective reporting, 
particularly for contaminated runways. Runway surface con- 
ditions may change rapidly according to precipitation, tem- 
perature, usage and runway treatment so actual conditions 
may differ significantly from those reported (FAA, 2006b). 
Icing conditions, too, also are known to be difficult to deter- 
mine even though they have an important impact on aircraft 
performance (Winn, 2006). 


The weather measured from ground stations may vary sig- 
nificantly from that experienced by the accident flight (Jerris 
et al., 1963), particularly if the weather station is located far 
from the accident location, although this is common only with 
very remote airports. Another difficulty lies in the dynamic 
nature of meteorological conditions. Wind strength and 
direction may change constantly during the course of an 
approach. It may not always be clear which reading is most 
relevant. Some judgment was necessary to enter the most 
appropriate reading into the database. 


Limited Data for Incidents. The importance of including 
data from incidents cannot be overemphasized. By excluding 
incident data, the project would not take into account poten- 
tially serious occurrences. However, a practical difficulty of 
incorporating incident data is the lack of it. The quantity 
and quality of incident data is in even greater doubt than for 
accidents. 


Most agencies provide information and reports for accidents 
and serious accidents only. Many countries have procedures to 
obtain information on incidents but, except for the United 
States, these reports are not readily available from Internet 
sources. For this study, the basic sources of data for nonserious 
incidents were the FAA AIDS and the NASA/FAA ASRS. 


One additional difficulty to incorporate the information 
into this project is the number of incidents reported. Some 
incidents are not reported because there were no conse- 
quences. To overcome this obstacle, a study was performed 
on the distribution of available data to assess the number of 
unreported incidents and to consider these cases when devel- 
oping the frequency and location models. A number of miss- 
ing incidents was assumed, as described in Appendix D, and 
a weighting factor was applied in the statistical analysis to 
develop the models. 


Additional Issues 


The database used for developing the final risk models in- 
cludes only those events that may challenge the RSA beyond 
the runway ends. The criterion utilized is similar to that used 
by the FAA (David, 1990) and includes those occurrences 
whereby the point of first impact or the final wreckage location 
is within 2000 ft from the threshold. Using such criteria, 459 
accidents and incidents were selected to compose the informa- 
tion that was used for developing the risk models. Table 2 sum- 
marizes the number and type of events by source of data. 

The main reason for the criterion applied is it would not be 
feasible to modify an existing RSA to more than 2000 ft in 
length, compared to the current 1000-ft standard. Most im- 
portantly, the additional safety benefit for having an RSA 
longer than 2000 ft certainly would be very small and not jus- 
tify the costs required for such improvements. Cases when 
the aircraft veered off the runway but did not challenge the 
area beyond the runway threshold also were removed, as 
these were out of the scope of this research. 

Data for events investigated by the NTSB were gathered 
from both investigation reports and the related dockets avail- 


Table 2. Summary of events utilized in this study. 


Database Source LDOR LDUS TOOR 
FAA AIDS (incidents) 14 29 12 
FAA/NASA ASRS (incidents) 79 11 9 
NTSB (accidents & incidents) 113 51 56 
TSB Canada (accidents) 23 1 5 
AAIB UK (accidents 24 (e) 5 
BEA France (accidents) 3 1 3 
Other (accidents) 18 0 2 
Total 274 93 92 
459 
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able at the NTSB library in Washington, D.C. Before data 
were gathered, database rules were developed to assure uni- 
formity for the information obtained by different researchers 
contributing to this project. 

Incident information was collected from NTSB, FAA AIDS 
and FAA/NASA ASRS databases. Accident data were obtained 
from NTSB and from aviation investigation agencies from 
other countries. 

A significant amount of aviation safety information is 
available worldwide, in many cases from specific websites. 
One of the main problems with this, however, is the frag- 
mentation of the information. Each agency has different 
search engines, and data are presented in different formats. In 
most cases, identifying the relevant events fulfilling the crite- 
ria for this project was quite challenging. 

Appendix E of this report presents the list of relevant acci- 
dents and incidents that fulfilled the criteria and filters estab- 
lished for the study and were utilized for developing the risk 
models. 


Supplementary Sources of Information 


Individual accident reports were evaluated to extract infor- 
mation. In addition, part of the data was complemented from 
other sources of information, particularly for aircraft, airport, 
and meteorological conditions. Based on the aircraft registra- 
tion, we have gathered information for aircraft involved in 
accidents and incidents from the following websites: 


e FAA REGISTRY N-Number Inquiry: 
http://registry.faa.gov/aircraftinquiry/NNum_inquiry.asp 

e US/World — Landings.com: 
http://www.landings.com/evird.acgi$pass*90705575!_h- 
www.landings.com/_landings/pages/search.html 

e Airframes.org - Passenger airliners, cargo airplanes, business 
jets, private aircraft, civil and military. 
http://www.airframes.org/ 

e Civil Aircraft OnLine Registers, Official Civil Aircraft 
Registers: 
http://www.airlinecodes.co.uk/reglinks.asp?type=Official 


Airport information, when not included in the incident or 
accident investigation reports, was obtained from other 
sources. Basically the following web sources were utilized in 
this study: 


e United States: AirNav provides detailed aeronautical infor- 
mation on airports and other information to assist pilots in 
gathering information for flight planning. Airport details 
include airport location, runway information, radio navi- 
gation aids, declared distances, and other information for 
pilots. http://www.airnav.com 
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e World: The World Aeronautical Database contains de- 
tailed, aeronautical information on nearly 10,000 airports 
and more than 11,000 Navigational Aids (NAVAID) 
worldwide. http://worldaerodata.com/ 


Many incident reports do not contain weather informa- 
tion, particularly when it is not deemed to be an important 
factor in the incident and was obtained from other sources. 
Weather for normal operations data also has been obtained 
from other sources, particularly from the National Oceanic & 
Atmospheric Administration (NOAA) database. NOAA is a 
federal agency focused on the condition of the oceans and the 
atmosphere. 

In many cases, particularly for accidents that occurred out- 
side North America, search engines available in the websites 
of accident investigation agencies are not very effective to fil- 
ter out those events that were irrelevant to this project. Some 
of the events were identified using these databases and, for a 
few cases, some accident data has been gathered from inde- 
pendent accident information websites. Two of the most used 
during this study included: 


e ASN Aviation Safety Database: The Aviation Safety Net- 
work is a private, independent initiative founded in 1996. It 
covers accidents and safety issues with regards to airliners, 
military transport planes, and corporate jets, and contained 
descriptions of more than 10,700 incidents, hijackings, and 
accidents. Most of the information are from official sources 
(civil aviation authorities and safety boards), including 
aircraft production lists, ICAO ADREPs, and country’s 
accident investigation boards. 

e World Aircraft Accident Summary: The World Aircraft 
Accident Summary (WAAS), produced on behalf of the 
British Civil Aviation Authority by Airclaims Limited, pro- 
vides brief details of all known major operational accidents 
worldwide. 


A typical example of this complementary information was 
the calculation of wind speed. Since the NTSB database con- 
tains wind speed and direction but not headwind and cross- 
wind components, determination of the orientation of the 
runway used by the accident aircraft allowed the research 
team to derive the headwind and crosswind components. 
Wreckage location often is described in words and required 
translation and interpretation to obtain estimates of location 
coordinates relative to runway centerline and thresholds. 


Accident/Incident Database Organization 


The accident and incident database was organized in 
Microsoft Access. The system provides some software tools 
that facilitate the use of the database in a flexible manner. The 


software includes facilities to add, modify, or delete data, 
make queries about the data stored, and produce reports 
summarizing selected contents. 

The database includes, for each individual event or opera- 
tion, the reporting agency, the aircraft characteristics, the 
runway and environmental conditions, result of the opera- 
tion (accident or incident), and other relevant information 
such as consequences (fatalities, accident costs) and causal or 
contributing factors and parameters required to develop the 
risk models. A unique identifier was assigned to each event, 
and the descriptions of each field and the database rules are 
available in Appendix F. The final database includes the cat- 
egories and fields listed in Table 3. 

Neither the NTSB nor the FAA routinely compiles data in 
this manner. Both agencies investigate accidents for aero- 
nautical purposes to determine ways to improve the design 
and operation of aircraft and airports and to foster better 
pilot skills and techniques. If land use factors are examined at 
all, it is incidental to the primary purpose of the investigation. 

As previously noted, it was difficult to gather information 
on incidents because they are rarely investigated to a level that 
could provide useful information for this study. Also, there 
are often few consequences associated with incidents. 


Normal Operations Data 


Another key approach in this study is the use of normal 
operations (nonaccident/nonincident flight) data for risk 
modeling. Various studies already have identified the lack of 
NOD as a major obstacle to the development of quantitative 
risk models (Department of Transport, 1979; Piers et al., 
1993; Khatwa et al., 1996; Khatwa and Helmreich, 1998; 
Eddowes et al., 2001; Li et al., 2001). The approach and the 
data utilized in this project were developed by Wong (2007). 

In the absence of information on risk exposure, even 
though the occurrence of a factor (e.g., contaminated run- 
way) could be identified as a contributor to many accidents, 
it is impossible to know how critical the factor is since many 
other flights also may have experienced the factor without in- 
cident. With NOD, the number of operations that experience 
the factor benignly, singly, and in combination can be calcu- 
lated, risk ratios can be generated, and the importance of risk 
factors quantified. This assessment may allow the prioritiza- 
tion of resource allocation for safety improvement (Enders 
et al., 1996). 

A large and representative sample of disaggregate U.S. 
NOD covering a range of risk factors has been collected, al- 
lowing their criticality to be quantified. The basic idea was to 
use these data and the information on U.S. incidents and 
accidents as a sample to develop the frequency models only, 
simply because the NOD represents only events occurring in 
the United States. The larger dataset comprising both U.S. 


Table 3. Database structure. 


Category 


Basic Info 


Aircraft Data 


Field Level 1 Field Level 2 
Accident ID 


Event ID 
Accident Class 


Event Type 
Researcher 


Source 
Country 


Location State 
City 

Date 

Time 

Basic Notes 

Make 

Model 

Series 

Serial Number 


Age No. of hours or Years 

No. of Engines 

Engine Type Turboprop, Turbofan (Low or 
High) or Turbojet 

Max Certified Landing 

Max Certified Takeoff 

Max Gross Weight 

Registration Number 

Regulations Reference 


ACFT Regulator 
Owner 


Operator 


Airport Data 


Consequences 


Code IATA Code 
Latitude 


Longitude 


Runway Number 
Landing Distance 


Available 
Takeoff Distance 
Available 


Landing Elevation 
Landing Latitude 


Landing Longitude 
Takeoff Elevation 


Takeoff Latitude 


Takeoff Longitude 
Runway condition 


Runway Grooved Yes/No 
ARFF Availability AtoF 

Control Tower Yes/No 
Temporary Yes/No 


Construction Works 


Runway Width 
Runway Slope 
Surface Material 


Paved Overrun Length 
Notes 


Aircraft Damage Destroyed, Substantial, Minor or 
None 
Change of Terrain Yes/No 


Consequence Area 


No. of Passenger Seats 
Total No. Of Seats 


Difficulty in Getting to Yes/No 
Wreckage 

Detailed Consequence 

Area 


(continued on next page) 
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Table 3. (Continued). 


Category Field Level 1 Field Level 2 
Aircraft Collision Status _Active/Passive/NA 
Visibility Min. 
Violation Tesla 
Approach Min. Yes/No 
Violation 
Approach Category Visual/Non-Precision/ILS Cat1, 2 
Required or 3 
Approach Category 
Used 


Other Aircraft Involved Yes/No 


Crash Controllability Fully/Partially/No 


Glide slope Captured Yes/No 
Go Around Yes/No 


GPWS Yes/No 
Sl ne - 
GPWS type 1 or 2™ Generation 


Localizer Captured Yes/No 
Runway Change Yes/No 
Stabilized Approach Yes/No 
Takeoff Aborted Yes/No 
Takeoff Aborted Speed 
Actual Weight at Crash 


Was Weight Estimated Yes/No 
Max Weight for 


Operation 
Destination Country 


Departure Country 


Detailed Info 


Diverted Flight Yes/No 
ELT Fitted and Yes/No 
Operational 

Flight Delayed Yes/No 
Flight Duration 

Flight Data Fuel Load 

Load Factor 

Operation Type 

Scheduled Yes/No 


Landing Distance 


Required 
Takeoff Distance 
Required 


Takeoff Weight 
Takeoff Fuel Load 


Weight restriction 
Violated Yecie 


Obstacle Depth 


Obstacle Height 
Hit Obstacles Obstacle Width 


Obstacle Location X, Y and Z 
Notes 


Terrain Depth 


Terrain Height 
Hit Terrain Terrain Width 


Terrain Location X, Y and Z 
Notes 


No. Passenger Injuries Fatal, Serious, Minor, None 


No. Plight Crew Fatal, Serious, Minor, None 


Injuries 
No. Cabin Crew Fatal, Serious, Minor, None 
Injuries 

Injuries No. Ground Crew Fatal, Serious, Minor 
Injuries 
On Ground Injuries Fatal, Serious, Minor 
Public Injuries Fatal, Serious, Minor 


Total Injuries 


Event Highest Injuries 
Notes 


Table 3. (Continued). 


Category Field Level 1 Field Level 2 
Ceiling 
Dew Point 
Electric Storm Yes/No 
Fog Yes/No 
Frozen Precipitation Yes/No 
Wind Direction 
Wind Velocity 
Wind Shear Yes/No 
Gusts 
Icing Condition Yes/No 
Weather Light Level Dawn/Day/Dusk/Night 
Rain Heavy/Moderate/Light/None 
Snow Yes/No 
Temperature 
Visibility 
RVR 
Actual Weather 
Different than Reported TeaiNo 
Weather General 
Local Variation Yes/No 
Tailwind 
Crosswind 
Explosion 
Fire 
No. Obstacles Hit 
Runway Exit Speed 
Total Wreckage Path 
Length 
Pilot Actively Avoided 
POFI Angle 
POFI Velocity 
POFI Location X,Y and Z 
Wreckage Location Longitude and Latitude 
Wreckage Location X,Y and Z 
Wreckage Info ene ee Exit X 
Runway Touchdown X 
Touchdown Speed 
Wreckage Site 
Elevation 
Height Above 
Threshold 
Approach Speed 
Oo} 
Week Fa LENE Up od segments 
Wreckage Slope Up to 4 segments 
Wreckage Surface Up to 4 segments 
Anomalies Power 
Brake (wheel brakes, spoilers or 
, reversers) 
Aircraft System Fault Tvdrulie 
Tire 
Other 
Weather Conditions Low Visibility 
Rain 
Wind Shear 
Tailwind 
Crosswind 
Gusts 


Low Ceiling 
Strong Wind 
Turbulence 


Freezing Rain 
Other 


(continued on next page) 
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Table 3. (Continued). 


Category 


Human Errors 


Runway Surface 


Conditions 


Field Level 1 


Field Level 2 
Incorrect Flight Planning 


Communication/Coordination 
Visual Illusion 

Fatigue 

Pressonitis 

Other 

Wet 

Contaminated - Standing water 
Contaminated - Rubber 
Contaminated - Oil 
Contaminated - Ice 
Contaminated - Slush 
Contaminated - Snow 
Contaminated - Paint 
Contaminated - Other 
Construction 


Down Slope 


Wildlife Hazards 


Approach/Takeoff 


Procedures 


Unstabilized - Low Approach 
Unstabilized - Low Speed 
Long Touchdown 
Unstabilized - High Speed 
High Above Threshold 


Takeoff Rejected 
Other 


Aircraft Body Type Wide or Narrow 


Aircraft Cost 


om Human Cost 


Total Event Cost 


and world occurrences was utilized to develop the location 
models. 

Incorporating risk exposure information into the accident 
frequency model enhances its predictive power and provides 
the basis for formulating more risk-sensitive and responsive 
RSA policies. Accident frequency models need no longer rely 
on simple crash rates based on just aircraft, engine, or opera- 
tion type. As discussed below, factors previously ignored by 
airport risk assessments and RSA regulations are accounted 
for using the models developed in this study. Moreover, this 
normal operations database can be used for future studies. 

The detailed source and sampling strategy of the NOD 
database is described in Appendix G. In addition, a small sam- 
ple of the NOD being utilized in this study is included in that 
section. A list of sampled airports is shown in Appendix H and 
the stratified sampling strata is presented in Appendix I. 

To derive the weights to be applied to each stratum, it was 
necessary to identify the relevant traffic from Terminal Area 
Forecasts (TAFs). Details on the calculation of TAF are pre- 
sented in Appendix J. 


Normalization of Data 


The small pool of relevant data available is a fundamental 
problem to risk assessment in aviation (Caves and Gosling, 
1999). Most studies have used data from different airports to 


2007 dollar value 
2007 dollar value 
2007 dollar value 


develop risk models. However, operation conditions and 
levels of risk are different at different airports. In addition, 
only raw distances between the final wreckage location and 
the runway end have been used to develop current FAA RSA 
recommendations. 

To mitigate this difficulty, information available for differ- 
ent airports was compared by using a normalization procedure, 
to transform existing data to a standard nominal airport (Kirk- 
land et al., 2003). To normalize aircraft accident data, the “nor- 
mal” airport is an airport situated at the International Standard 
Atmosphere (ISA) conditions, with level surrounding terrain 
or obstacles and an infinitely long, hard runway. 

Normalization was conducted for the effects of terrain on 
wreckage location using the models developed by Kirkland et al. 
(2003) and the effects of the local atmospheric conditions on 
the aircraft’s performance, based on standard corrections for 
aircraft distance required. Major factors that affect the runway 
distance required for the operation and used in the flight man- 
ual calculations are runway slope, runway elevation, and the air 
temperature; however, a correction was not applied to the slope 
in the RSA due to missing information for the majority of the 
events. The RSA slope is indeed an important factor on the 
wreckage distance and this information should be collected and 
made available in incident and accident investigation reports. 
Normalization procedures used in this study are presented in 
Appendix K. 


Ideally, normalization procedures should consider the dis- 
tance required relative to the runway distance available dur- 
ing the operation. However, the attempt to incorporate this 
factor proved difficult due to a lack of available information 
to compute the distances required, for both the accident data 
and NOD. 

In addition, some factors cannot be normalized, such as 
the pilot’s skill or differences in safety records between coun- 
tries. Despite these difficulties the technique utilized in this 
study creates a larger pool of relevant and comparable data 
for sound model building. Results from models built with 
normalized accident and normal operations data then can be 
applied to specific airports through denormalization. 


Development of Risk Models 


In this study overrun and undershoot risk models were de- 
veloped to allow the analysis of RSAs in the light of specific 
factors related to existing operation conditions. The basic 
concept was to model the probability distribution for wreck- 
age location in the proximity of the runway threshold. The 
concept is illustrated in Figure 3 for overruns and in Figure 4 
for undershoots. 
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The illustration for overrun incidents shows a typical prob- 
ability distribution for the stopping location during a landing 
or rejected takeoff operation. (The illustration is not to scale 
and is only intended to help understanding the concept.) The 
great majority of aircraft will stop within the runway bound- 
aries, represented by the lightly shaded area of the probabil- 
ity distribution. However, in a few cases, the aircraft may not 
be able to stop before the runway end and will stop on the 
RSA or even beyond. This probability of overrunning 
the runway is represented by the dark shaded area. It would 
be best to model the whole probability distribution for the 
aircraft stopping location, but this information is not avail- 
able for NOD, and an alternative modeling approach was re- 
quired in a two-step process: evaluating the probability an 
aircraft will in fact overrun the runway, and modeling the 
likelihood the aircraft will stop beyond any given distance 
from the runway threshold. 

The same concept was used for the point of first impact 
(POFI) during landing undershoots. Figure 4 depicts the 
probability distribution for the touchdown location. Again, 
for the great majority of operations, the aircraft will touch- 
down within the runway boundaries, represented by the 
lightly shaded area of the probability distribution. The dark 
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Figure 3. General concept for modeling aircraft overruns. 
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Figure 4. General concept for modeling aircraft undershoots. 


16 


bx” 


P {Location > y}= e— 


Figure 5. General concept for modeling aircraft lateral deviation. 


shaded area represents the probability the aircraft will land 
before the approach end of the runway. 

In addition, it was necessary to model the probability 
distribution for the aircraft wreckage location relative to the 
extended runway centerline. The concept is represented in 
Figure 5. Using the “x” and “y” models combined will allow 
users to evaluate RSA off standard dimensions and overall 
configuration. The distribution only represents the location 
distribution for aircrafts overrunning or undershooting the 
runway. 

An initial analysis of the database and NOD was required to 
determine which risk factors were available to be built into the 
parametric models. The generic model and the wider insight 
gained from the database allowed an initial generic estimate of 
risk for a given situation to be made and identification of 
whether there were other significant risk factors that applied. 
The three-part modeling approach was built for each accident 
type, as represented in Figure 6. Before developing the final 
model structure, the available data were evaluated statistically, 
in order to develop the proper model structure and to ensure 
the parameters were compatible with model assumptions. 


Approach Elements 


Event Probability. The likelihood of an aircraft overrun 
or undershoot accident or incident depends on the operation 
conditions, including airport characteristics, weather condi- 
tions, and aircraft performance. 

The probability of an accident per movement—the accident 
rate—is determined from historical data on numbers of move- 


ments carried out at reference airports and the number of acci- 
dents that occurred during those movements. Loughborough 
University has pioneered the use of NOD to include a multitude 
of risk factors in the assessment of accident probability. Using 
this approach allowed a far more discriminating analysis than 
relying solely on the accident rate. As a result, conclusions on 
RSA risks will reflect better the actual conditions and circum- 
stances of specific airports. 


Accident Location. In reality, the probability of an acci- 
dent is not equal for all locations around the airport. The 
probability of an accident in the proximity of the runways is 
higher than at larger distances from the runway. This de- 
pendence is represented by the accident location model, 
which is the second main element of the current methodol- 
ogy. The accident location model is based on historical data. 
The distribution of accident locations relative to the runway 
will be modeled through statistical functions introduced by 
Eddowes et al. (2001). By combining the accident location 
model with the accident probability, the local probability of 
an accident can be calculated for each runway end. 


Accident Consequences. The consequences of an acci- 
dent are a function of the dimensions of the actual RSA, of the 
aircraft and impact parameters (such as aircraft size, quantity 
of on-board fuel, impact angle, etc.), and of the local type of 
terrain and obstacles. The size of the accident area is not equal 
for every airport or area within the airport. The influence of 
the aircraft and impact parameters and the type of terrain on 
the size of the consequence area, as well as the lethality and 
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Figure 6. Modeling approach. 
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damage of the consequences, are defined in the consequences 
model, the third main element in the current methodology. 
For this purpose, lethality is defined as the actual probability 
of being killed within the consequence area. Aircraft damage 
is translated as the direct cost of property loss for an accident. 


Accident/Incident Probability Model 


To examine the accident propensity associated with differ- 
ent factors (e.g., environmental conditions), logistic regression 
was used to develop statistical models for accident/incident 
occurrence probability. 

A number of numerical techniques could be used to carry out 
the multivariate analysis, but logistic regression was preferred. 
First, the technique is suited to models with a dichotomous out- 
come (incident and nonincident) with multiple predictor vari- 
ables that include a mixture of continuous and categorical pa- 
rameters. Logistic regression also is appropriate for case-control 
studies because it allows the use of samples with different sam- 
pling fractions, depending on the outcome variable without giv- 
ing biased results. In this study, logistic regression allowed the 
sampling fractions of accident flights and of normal flights to be 
different. This property is not shared by most other types of re- 
gression analysis (Nagelkerke et al., 2005). 

Backward stepwise logistic regression was used to calibrate 
the three frequency models because of the predictive nature 
of the research. This technique is able to identify relationships 
missed by forward stepwise logistic regression (Hosmer and 
Lemeshow, 2000; Menard, 2001). Due to the more stringent 
data requirements of multivariate regression, cases with miss- 
ing data were replaced by their respective series means. 

To avoid the negative effects of multi colinearity on the 
model, correlations between independent variables were 
tested first to eliminate highly correlated variables, particu- 
larly if they do not significantly contribute to explaining the 
variation of the probability of an accident. 

The basic model structure selected for this study is in the 
following form: 


1 


P{ Accident _ Occurence} = 
1 + ehh X1+b2X2413X3+... 


(1) 


where 


P{Accident_Occurrence} = the probability (0-100%) of an 
accident type occurring given 
certain operational conditions; 

X;=independent variables (e.g., 
ceiling, visibility, crosswind, 
precipitation, aircraft type); and 

b; = regression coefficients. 


The use of NOD in the accident frequency model provided 
a major improvement in the modeling of accident occur- 
rence, as discussed previously. The analysis with NOD also 
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adds to the understanding of cause-result relationships of the 
two accident types. This constitutes a causal element in the 
risk models, so the modeling tool developed can be used to 
assess risk reduction strategies and estimate future risk levels, 
given trends in influential factors in an airport context. 

A previous Loughborough University study on overruns 
found that the model developed for landing overrun risk using 
NOD on excess landing distance available is 22 times more 
predictive than models based on flight type alone (Kirkland, 
2001). Additional analyses using NOD have been conducted 
since, and they continue to show the importance of assessing 
the criticality of risk factors beyond the simple accident/ 
movement rate (Wong et al., 2005b, 2006). 


Accident/Incident Location Model 


The model structure selected for accident location was 
used by Eddowes et al. (2001) and is in the following form: 


P{Location > x}=e~*" (2) 
where 


P{Location > distance} =the probability the overrun/ 
undershoot distance along the 
runway centerline beyond the 
threshold is greater than x; 

x =a given location or distance be- 
yond the threshold; and 
a, n= regression coefficients. 


This dependence is represented by the accident location 
model, which is the second main element of the current 
methodology. The accident location model is based on his- 
torical data on accident locations. The distribution of acci- 
dent locations relative to the runway was modeled through 
statistical functions. By combining the accident location 
model with accident probability, the local probability of an 
accident can be calculated for each runway end. 

When plotting the percentage of accidents where the air- 
craft stopped beyond a certain distance from the threshold, 
in case of overruns, or first impacted the terrain, for under- 
shoots, the probability diminishes the greater the distance is, 
as depicted in Figure 7. 

The probability and location models will provide a quantita- 
tive assessment based on operating conditions for a specific 
airplane landing or taking off at a specific runway. In addition, 
it is necessary to relate these probabilities with the RSA condi- 
tions to provide an assessment of the probability that the con- 
sequences of an incident are severe. This is the final component 
of the approach, described in the following section. 


Consequence Model 


The consequences modeling approach should provide a 
qualitative assessment of the severity of an accident, based on 
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Figure 7. Typical trend for wreckage location 
model. 


the location model and the existing runway characteristics, to 
include dimensions of existing RSA, airplane weight, location 
and type of obstacles, and topography of surrounding terrain. 
The approach used in this project was to model the probability 
of severe consequences using the frequency and location mod- 
els, coupled with existing RSA configuration and obstacles. 

The consequences of an accident depend on several factors 
that are difficult to model, such as the energy of the crash 
(speed, aircraft weight, and size), quantity of fuel and occur- 
rence of fire after impact, type of obstacle (height, depth, ma- 
terial, size), impact angle, and the local type of terrain. Initial 
attempts to model consequences focused on the relationship 
between the raw or normalized distances and the severity of 
the accident, reflected by the amount of damage and cost of 
injuries. The overall consequences of the accidents were quan- 
tified by the total direct costs for injuries and aircraft damage. 

The approach proved difficult to implement because the re- 
lationship between accident location and consequences was 
poor. In many situations the consequences were related to the 
speed that the aircraft hit an obstacle and the type of the obsta- 
cle. Information for the former was not available for the great 
majority of cases. Therefore, the efforts to model consequences 
were directed to providing a rational approach that incorpo- 
rated the location model. The basic idea is simple but effective. 
The higher the speed, and hence the energy when an aircraft 
hits an obstacle, the greater the consequences. The sturdier the 
obstacle, the greater the consequences. The larger the obstacle, 
the greater the probability the aircraft will hit the obstacle. 

The speed of the aircraft striking the obstacle is related to 
the distance the aircraft would take to stop if no obstacles 
were present in the area adjacent to the runway ends. Based 
on the location model, the terrain type, and the deceleration 
model developed by Kirkland (2001), the probability the air- 
craft moving above a certain speed when hitting the obstacle 
can be estimated. 


The speed necessary to cause significant damage to the air- 
craft and potentially severe consequences should be judged 
based on the type of aircraft, type of terrain, and type of obsta- 
cle. For this study, only general recommendations are provided 
to assess the interaction between the aircraft and obstacles. 
Using some simple assumptions it is possible to evaluate the 
overall risk of severe consequence accidents. 

One of the difficulties posed to evaluate consequences of 
accidents was to integrate the number and severity of injuries 
with the property loss. It was not possible or practical to eval- 
uate indirect consequences such as lost revenue, lost work 
time, disruption of flight schedule, and negative customer re- 
action to accidents. 

In this study, estimates for direct costs of accidents are pro- 
vided as a means to integrate personal injury and property 
loss. Although it is estimated that indirect costs typically rep- 
resent four times the value of direct costs, only the latter will 
be used in this study. The parameters that were evaluated 
include the cost of the accidents, and number and type of 
injuries. The relationships between these consequence 
parameters and potential independent variables include 
the wreckage path length, the number of obstacles hit during 
the accident, the location of these obstacles, and land use type 
for the area beyond the existing RSA. 


Accident Costs 


The consequences of accidents, as documented in investi- 
gation reports, are described in terms of the number of 
injuries and the level of damage to the aircraft. Although 
third-party injuries also were accounted for in this study, 
property loss not related to aircraft damage was not evaluated 
for the lack of information. Injuries are classified according 
to ICAO criteria into four groups: none, minor, serious and 
fatal. The number of passengers and crew members for each 
level generally is available in the accident reports. Based on 
the total number of passengers/crew on board the fatality rate 
for each accident was calculated. Damage to aircraft also is 
described according to four classification groups: none, 
minor, substantial, and destroyed. 

In addition to the raw classification, direct cost of acci- 
dents was calculated based on the number and type of 
injuries, as well as the damage to the aircraft. The basic 
source for accident cost is the Guide for Economic Values for 
FAA Investment and Regulatory Decisions (GRA, 2004). The 
objective of this report is to present a set of cost items and 
quantify the specific values recommended that FAA use in 
future regulatory evaluations in the conduct of benefit-cost 
and other evaluations of investments, including certain 
Airport Improvement Program (AIP) grants, and regula- 
tions subject to FAA decision making. They also are used by 


others, including airports, in benefit-cost analysis of pro- 
posed investments. The basis for estimating the direct costs 
is presented in Appendix L. 


Development of Prototype Software 
for Risk Analysis 


As part of this research study, prototype software that in- 
corporates and integrates the risk models was developed. 
Users may input raw data and obtain normalized data, run a 
risk assessment analysis, and obtain denormalized results. 
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The software serves as a tool for risk assessment associated 
with overrun and undershoot accidents and provides a basic 
yet useful format for risk analysis professionals to assist air- 
port operators in evaluating RSAs. 

Input data include the airport information, target level of 
safety (TLS), RSA characteristics including dimensions and 
type of terrain, and multiple historical or planned operations 
that may challenge the RSA in the event of overrun or under- 
shoot incident. Output includes frequency distribution of 
risks for each type of accident and the percentage of flights 
subject to risk above a TLS. 
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CHAPTER 3 


Findings and Applications 


Database Summary Statistics 


This section provides a statistical description of the data 
gathered in this study and included in the ACRP 4-01 acci- 
dent and incident database. The final database incorporates 
459 accidents and incidents. Figure 8 depicts the distribution 
of events by type of accident and incident (LDOR, LDUS, or 
TOOR). 

Proportionally, there have been more LDOR than LDUS 
and TOOR, as shown in Figure 9. The numbers of LDUS and 
TOOR are similar. 

Although some events prior to 1982 were included in the 
study, the majority of cases date from 1982 to 2006. The dis- 
tribution of incidents and accidents is variable along the 
period data was collected. However, in the average (from 
1984 to 2004), the number of reported accidents and inci- 
dents was similar and averages 18 events per year (9 accidents 
and 9 incidents). Part of the reduction observed for 2005 to 
2007 is due to the unavailability of the reports when the data 
for this study were collected. For many events during this 
period, either the reports had not been completed or they 
were not yet available in electronic format. Figure 10 sum- 
marizes the number of events per year. 


Summary of Anomalies Associated 
with Accidents and Incidents 


An FHA was conducted during the initial stages of this 
study to identify the relevant factors associated with aircraft 
overrun and undershoot events so that data on these param- 
eters could be gathered and included in the accident database 
and be used for developing risk models. 

The majority of investigation reports describe causal and 
contributing factors to accidents but in general these are not 
reported for incidents. In addition, certain factors not de- 
scribed as causal or contributing factors in the accident are 
relevant to the present study. For example, a runway overrun 


investigation report may describe high approach speed and 
long touchdown as causal factors, but the report described 
the runway surface as wet. Although the latter was not con- 
sidered a relevant factor in the investigation, the anomaly was 
present and is included in the summary statistics that follow. 

The anomalies were divided into six different categories to 
aid in understanding the factors leading to aircraft overrun 
and undershoot events: 


e Aircraft System Fault (SysF); 

e Wildlife Hazards (WH); 

e Weather Conditions (W); 

e Human Errors (H); 

e Runway Surface Conditions (R); and 
e Approach/Takeoff Procedures (AT). 


Several anomalies within each of these categories may be 
present during accidents and incidents. The majority of these 
anomalies were taken from the list of causal and contributing 
factors described in the investigation reports. In a few cases, 
even when not listed in the report, if an additional anomaly 
was identified, it was included in this analysis. For example, 
some investigation reports did not describe the wet runway 
as a causal or contributing factor to the accident, but rain 
during touch down on the runway was listed, and wet runway 
was included in the analysis. 

The complete list of anomalies within each of the above 
categories and used in this study is shown in Table 4. 

Figure 11 depicts the distribution by category for landing 
overruns. In this case, anomalies are mostly related to weather, 
human error, runway conditions, and approach procedures. 

Figure 12 shows the frequency of anomalies by the category 
for undershoots. Similarly to landing overruns, the anomalies 
are mostly related to weather, human error, runway condi- 
tions, and approach procedures. Except for runway conditions, 
the incidence of anomalies was higher for the accidents under 
the predominant categories. 
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Figure 8. Distribution of events by type. 


A summary of anomalies by category for takeoff overrun 
events is presented in Figure 13. For most of the events there 
were anomalies in the takeoff procedures. When there were 
anomalies related to weather conditions there were signifi- 
cantly more events in the accident category than incidents. 
The same conclusion is generally true for human errors, while 
system faults are mostly related to incidents. 

Anomalies reported or identified were included within each 
of these categories, as shown in Figure 14 for LDOR events. 
Only the anomalies having more than 10 percent incidence are 
reported here, but a comprehensive list of anomalies is avail- 
able in the accident/incident database. 

The highest incidence anomalies for landing overrun are 
contaminated and wet runways. Sometimes these anomalies 
occur in combination (e.g., a wet runway contaminated with 
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Figure 9. Distribution by type of event. 


rubber). For contaminated runways, ice was the most pre- 
dominant contaminant in the accidents and incidents evalu- 
ated. Three additional factors with high incidence for landing 
overruns are long touchdown, high speed during the ap- 
proach, and the presence of rain. 

According to the numbers presented in Figure 15 for 
LDUS, the most frequent anomaly was low visibility, followed 
by rain, particularly for the accidents. Gusting conditions had 
high incidence for these accidents. As expected, approaches 
below the glide path are an important anomaly for this type 
of event. Visual illusion was a significant factor only for land- 
ing undershoots. 

The presence of rain, gusting, crosswind, and low ceiling 
conditions were most predominant for these accidents when 
compared to incidents. 

Figure 16 depicts the most frequent anomalies for TOOR 
events. As expected, rejecting the takeoff operation at high 
speeds led to the majority of accidents and incidents. The sec- 
ond most important anomaly was incorrect planning, such 
as: aircraft overweight, short takeoff distance available, and 
incorrect load distribution in the aircraft. Basically, the fac- 
tors are equally frequent for accidents and incidents, except 
for the presence of rain, gusting, and crosswind conditions. 
These were more important for accidents when compared to 
incidents. 

A summary of the most frequent anomalies for all events 
by accident type is shown in Table 5. The “X” represents the 
anomaly was present in more than 10 percent of the cases for 
the specific event type: LDOR, LDUS, or TOOR. 


Unreported Events 


When using U.S. accidents and incidents as a sample, the 
number of reported incidents (53 percent) is close to the 
number of accidents (47 percent), when it was expected to see 
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Figure 10. Events per year. 


a much higher number of incidents compared to accidents. 
One possible explanation for this phenomenon would be that 
some incidents are unreported. Therefore, an analysis of the 
number of unreported incidents was carried out. 

The methodology for evaluation of under reporting inci- 
dents is based on the assumption that there is a progressive 
decrease in the probability of travel to any given distance 
from the runway end with increasing distance. Such behavior 
is evident from the empirical accident and incident data set 
and is consistent with theoretical considerations of the nature 
of the event. The same basis considerations apply to LDOR, 
TOOR, and LDUS. Behavior of this type can be represented 
by a cumulative probability distribution function of the fol- 
lowing form: 


PROB{d > x}=e-™" (3) 


where PROB{d > x} is the probability of traveling a distance 
d greater than x. 


Where there is full reporting of events it is expected that the 
available empirical data should fit a function consistent with 
this basic form. Where there is under-reporting, some dis- 
tortion in the apparent behavior can be expected. Failure to 
report is expected to be more likely for events where the dis- 
tance traveled off the runway is relatively low. The reported 
cumulative probability distribution (CPD) will be depressed 
at lower values of x but co-incident with the full distribution 
at higher values, as shown schematically in Figure 17. 

There may be other factors that distort the form of the 
CPD. A data set of incidents is expected to lack a dispro- 
portionate number of events at greater distances, since 
these would be expected to more likely result in more seri- 
ous consequences and be classified as accidents. On that 
basis, it is likely to be most appropriate to apply the analy- 
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sis method proposed to a data set of incidents and accidents 
combined. 

Another factor that might distort the form of the curve is 
the obstacle environment beyond the runway end. An obsta- 
cle might cause the aircraft to come to a stop earlier than it 
might otherwise. Generally, there is an increased probability 
of an aircraft encountering an obstacle the farther it has trav- 
eled, and particularly when it has traveled farther than the 
RSA, this effect will lead to a reduced probability of aircraft 
traveling to greater distances than would otherwise be the 
case. The implications of this phenomenon required further 
consideration as part of this analysis. 

The analysis of unreported incidents is presented in Ap- 
pendix D, and the results are summarized in Table 6. Based 
on these numbers, different weights for statistical modeling 
were used to reflect the expected rate of incidents relative to 
accidents. 


Probability of Incident-Frequency 
Models 


The chance of an aircraft overrunning or undershooting a 
runway depends on the probability of accident per aircraft 
movement and the number of movements (landings and 
takeoffs) carried out per year. 

Logistic regression, discriminant analysis, and probit 
analysis were evaluated for modeling the probability of air- 
craft overrun and undershoot events. Discriminant analysis 
was not used because it involves numerous assumptions, 
including requirements of the independent variables to be 
normally distributed, linearly related, and to have equal vari- 
ance within each group (Tabachnick and Fidell, 1996). 
Logistic regression was chosen over probit analysis because 
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Table 4. Anomalies during aircraft overrun 
and undershoot events. 


Category 


Anomaly Type 


Aircraft System Fault 


Tire 


Hydraulic 


Power 


Brake 


Other 


Wildlife Hazards 


Bird strike 


Other 


Weather Low Visibility 


Wind Shear 
Tailwind 


Crosswind 


Gusts 


Low Ceiling 
Strong Winds 


Turbulence 


Freezing Rain 


Rain 


Other 
Fatigue 


Human Error 


Communication/Coordination/ 


Planning 


Pressonitis 
Visual Illusion 
Other 
Runway Surface Wet 
Contamination / Low friction _ Standing Water 
Rubber 
Oil 
Slush 
Snow 
Ice 
Paint 
Construction 
Downslope 
Other 
Approach/Takeoff Unstabilized Approach Approach Below Flight 
Procedures Path 
Approach Above Flight 
Path 
High Speed 


Low Speed 


Long Touchdown 


Takeoff rejected at high speed 


Other 


the latter does not give the equivalent of the odds ratio and 
changes in probability are harder to quantify (Pampel, 2000). 

Logistic regression is suited to models with a dichotomous 
outcome (incident and nonincident) with multiple predictor 
variables that include a mixture of continuous and categori- 
cal parameters. Logistic regression also is appropriate for 
case-control studies because it allows the use of samples with 
different sampling fractions depending on the outcome vari- 
able without giving biased results. In this study, it allowed the 
sampling fractions of accident flights and normal flights to be 
different. This property is not shared by most other types of 
regression analysis (Nagelkerke et al., 2005). 

Backward stepwise logistic regression was used to calibrate 
the three frequency models because of the predictive nature 
of the research. The selected technique is able to identify 


relationships missed by forward stepwise logistic regression 
(Hosmer and Lemeshow, 2000; Menard, 2001). Due to the 
more stringent data requirements of multivariate regression, 
cases with missing data were replaced by their respective 
series means. 

Every risk factor available in both Accident/Incident data- 
base and NOD were used to build each model. Table 7 shows 
the final parameters retained by the backward stepwise logis- 
tic regression as relevant independent variables for each of the 
frequency models. 

It should be noted that it was not possible to include some 
risk factors in the frequency models, for example, the ratio 
between the landing distance available and the landing 
distance required. Although a possible important factor to 
assess runway criticality, the lack of information for landing 
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Frequency Distribution of Anomalies for Landing Overruns 
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Figure 11. Frequency of anomalies by category (LDOR). 


distance required in the normal operations data precluded 
the use of such variables in the frequency models. However 
it is difficult to evaluate how much improvement such a fac- 
tor would bring to the model accuracy. Theoretically the 
runway length always should be compatible with the dis- 
tances required by the aircraft under certain conditions. In 
this sense the new factor may bring little benefit to the model 


but, on the other hand, a larger safety factor for distances 
required also should be expected for most flights operating 
in longer runways. 

The goal was to develop risk models based on actual ac- 
cidents/incidents and normal operation conditions so that 
the probability of occurrence for certain conditions may be 
estimated. The use of such models will help evaluate the 
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Figure 12. Frequency of anomalies by category (LDUS). 


Frequency Distribution of Anomalies for Takeoff Overruns 
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Fault Condition Conditions Procedures 
Figure 13. Frequency of anomalies by category (TOOR). 
likelihood of incident occurrence for a runway that is sub- P{Accident_Occurrence} = s the probability (0-100%) of an 


ject to certain environmental and traffic conditions over 
the year. 
The frequency model is in the following form: 


P{ Accident _ Occurrence} = =e (4) 


1 + em (b0+b1X1+b2X2+03X3+...) 


where 


Landing Overrun - Most Significant Anomalies 


% of Total for Event Type 


W- 


[ Fr x 
rT 


accident type occurring given 
certain operational conditions; 
X; = independent variables (e.g. ceil- 
ing, visibility, crosswind, tail- 
wind, aircraft weight, runway 


condition, etc.); and 
b; = regression coefficients. 


TDown 


| | ie 

SysF- W- W- ~ 

Othe Rain Xwind | Gusts 
6.1 


2% | 31.5% i : 16.1% 


Anomaly 


Figure 14. Most frequent anomalies (LDOR). 
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Landing Undershoot - Most Significant Anomalies 


% of Total for Event Type 
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Figure 15. Most frequent anomalies (LDUS). 


Before logistic regressions were performed, it was ensured 
that all assumptions for the statistical procedure were met. 
Logistic regression is relatively free from assumptions, espe- 
cially compared to ordinary least squares regression. How- 
ever, a number of assumptions still apply. One of these is a 
linear relationship between the independents and the log 
odds (logit) of the dependent. 

The Box-Tidwell transformation test was used to check 
whether all continuous variables met this assumption 
(Garson, 1998). This involved adding to the model interac- 


tion terms that are the cross-product of each independent 
variable times its natural logarithm [(X)In(X)]. The logit lin- 
earity assumption is violated if these terms are significant. In 
the current analysis, the continuous variables were found to 
have non-linear logits. As a solution, these variables were 
divided into different categories according to standard equal 
intervals using landing NOD and accident data. The vari- 
ables then were converted into categorical ones with these 
different levels, each being a separate logit independent 
variable. 
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Figure 16. Most frequent anomalies (TOOR). 
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Table 5. Summary of anomalies for aircraft overruns 


and undershoots. 


Anomaly LDOR LDUS TOOR 
ACC INC ACC INC ACC INC 

Brake system failure xX x 
Power failure xX 
Tire failure xX 
Other aircraft system fault x X xX 
Rain xX X xX x x 
Low Visibility 4 x x x x 
Low Ceiling xX 
Tailwind xX * 
Crosswind X X X x 
Wind shear x 
Gusts Xx X xX 
Improper flight planning x xX x X x X 
Visual illusion x 
Other human errors xX xX X X 
Wet runway xX x x 
Contaminated runway xX x 
Long touchdown xX xX 
High speed during approach xX xX 
Low speed during approach xX 
Approach too low x xX 
Other approach anomalies x 
Runway construction xX 
Rejected takeoff at high speed x xX 


A test for multicollinearity is required for multivariate 
logistic regression. Collinearity among the predictor variables 
was assessed by conducting linear regression analyses to ob- 
tain the relevant tolerance and Variance Inflation Factor (VIF) 
values. None of the tolerance values were smaller than 1, and 
no VIF value was greater than 10, suggesting that collinearity 
among the variables is not serious (Myers, 1990; Menard, 
2001). Kendall’s Tau also was used to assess potential correla- 
tions between predictor variables that are likely to be related. 

Two pairs of variables had Kendall’s Tau correlation coeffi- 
cient between 0.51 and 0.60, indicating moderate correlation: 
equipment class with airport hub size and icing conditions 
with frozen precipitation. Since none of the correlations were 


serious, all variables were kept in the multivariate model, and 
caution was applied in interpreting the results. This is preferred 
to the alternative solution of removing variables, which would 
lead to model misspecification. 

Although the R? for the models ranged between 0.148 and 
0.245, as shown in Table 8, relatively low values are the norm 
in logistic regression (Ash and Schwartz, 1999), and they 
should not be compared with the R? of linear regressions 
(Hosmer and Lemeshow, 2000). The analysis of models using 
Receiver Operating Characteristic (ROC) curves to classify 
flights as “accident” or “normal” suggests good to excellent 
classification accuracy for such models (C-Statistic from 
0.819 to 0.872). 


Form of Cumulative Probability Distribution 
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Figure 17. Schematic form of cumulative distribution 


functions. 
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Table 6. Summary results for under-reported incidents. 


Total # of Total # of % Unreported Estimated # 
Accidents and Incidents Incidents ** Unreported 
Incidents Incidents 
LDOR 240 28.8% 17 
LDUS 81 9.6% 7 
TOOR tS 28.8% * 1 


Note: * value assumed based on comparisons with LDOR 
** based on incidents occurring at small distances from threshold 


Due to the case-control set-up of the study, the constant 
(intercept) term bp of the final formula must be adjusted to 
account for the different sampling fractions between the cases 
and the controls. The following formula was used for this 
purpose (Hosmer and Lemeshow, 2000): 


Bs fd In(t,/to) ale bo (5) 
where 


b*, = the original intercept, 
t, = the sampling fraction of cases, 
tf) = the sampling fraction of controls, and 
by =s the adjusted intercept. 


Although parameter t, is normally one when relevant in- 
formation is available for all events, it was necessary to adjust 
these values to reflect under-reporting of incidents. 

From the NOD sampling exercise, it was calculated that the 
total number of relevant normal operations from 2000 to 2005 
inclusive is 191,902,290. That is 44.78 percent of the period’s 
total itinerant operations excluding military operations. From 
the TAF, the total number of itinerant operations from 1982 
to 2002 inclusive (the accident sampling period) excluding 
military operations was computed to be 1,408,495,828 move- 
ments. Of the latter, 44.78 percent equates 630,792,133 move- 
ments. A detailed description on the calculation of relevant 
terminal area forecast traffic is presented in Appendix J. Since 


Table 7. Independent variables used 
for frequency models. 


Variable LDOR LDUS 

Aircraft Weight/Size Xx 

Aircraft user class 

Ceilin 

Visibility Xx 

Fog 

Crosswind Xx 

Gusts 

Icing Conditions xX 

Snow X xX 

Xx 
X 


=| 
° 
ie} 
=] 


tal 
PA PSI | PPS 


PAIRS} [PS [PS 1S] PS 1S |S 


Rain 
Temperature Xx 
Electrical Storm 


Xx 
XxX 
Turboprop/Jet XxX 
xX 
4 


~* 


Foreign Origin/Destination 
Hub/Non-hub airport 


the total sampled normal operation population is 242,420 
flights, 


ty = 242420/630792133 = 3.843 x 107 


With t, and t), the adjusted intercepts of each of the risk 
model formula can be calculated: 


Bo = In(t)/to) tr bo = In(#,/3.843 x 10-*) P bo = 7.864 + bo (6) 


Where b*) is the original intercept, t, is the sampling frac- 
tion of cases, ty is the sampling fraction of controls, and by is 
the adjusted intercept. The calculated parameters for each 
model are shown on Table 9. 

Using the adjusted intercepts, the final frequency models 
are the following: 


Landing Overrun (7) 


b=—15.456 + 0.55 1(HeavyAcft) — 2.113(CommuterAcft) 
—1.064( MediumAcft) —0.876(SmallAcft) 
+0.445(TurbopropAcft )— 0.857( ForeignOD) 
+1.832(CeilingHeight < 1000 ft) 
+1.639(CeilingHeight1001— 2500 ft) 
+2.428( Visibility < 2SM)+1.186(Visibility2—4SM) 
+1.741 (Visibility 4—6SM)+0.322(Visibility6—8SM) 
—0.532(Crosswind2 —5knts)+1.566(Crosswind5 — 12knts) 
+1.518(Crosswind > 12knts) +0.986(ElectStorm) 
+1.926(IcingConditions) + 1.499(Snow) —1.009(Temp <5C) 
—0.631(Temp5 —15C)+0.265(Temp > 25C) 
+1.006( NonhubApt) + 0.924(SignificantTerrain) 


Table 8. Summary statistics 
for frequency models. 


Model Roo (8 

LDOR 0.245 0.872 
LDUS 0.199 0.819 
TOOR 0.148 0.861 


Table 9. Calculated model intercepts. 


Type of Sampling Original Adjusted 
Event Fraction (t;) Intercept (b*o) Intercept (bo) 
LDOR 0.938274 -7.656 -15.45637 
LDUS 0.943765 -7.158 -14.96421 
TOOR 0.997447 -8.790 -16.65153 


Landing Undershoot (8) 


b=—14.9642 + 0.036(HeavyAcft) —1.699(CommuterAcft) 
—0.427( MediumAcft)+1.760(SmallAcft) 
+0.288(UserClass1) + 0.908(UserClass2) —1.042( ForeignOD) 
+0.199(CeilingHeight < 1000 ft) 
+1.463(CeilingHeight 1001 — 2500 ft) 
+2.074( Visibility < 2SM)+0.069( Visibility2 —4SM) 
—0.185(Visibility4 — 6SM ) —0.295(Visibility6 —8SM ) 
+1.830( Fog) —1.705( Rain) —0.505(Temp <5C) 
—0.874(Temp5 —15C)—0.446(Temp > 25C) 
+2.815(Icing) + 2.412(Snow) 
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Takeoff Overrun (9) 


b=—16.6515+0.721(HeavyAcft) —0.619(CommuterAcft) 
—0.009(MediumAcft) +1.669(SmallAcft) +1.336(UserClass1) 
+1.052(UserClass2) + 1.225(CeilingHeight < 1000 ft) 
+1.497(CeilingHeight1001— 2500 ft) 
+0.201( Visibility < 2SM)—1.941(Visibility2—4SM) 
—0.366(Visibility4 —6SM)+0.317(Visibility6 — 8SM) 
+1.660( Fog) —0.292( Xwind2 —5knts) 
+1.598( Xwind5 —12knts)+1.781( Xwind > 12) 
~0.536(Temp <5C)—0.507(Temp5—15C) 
+0.502(Temp > 25C) +1.805(Icing) + 2.567(Snow) 


Where: 

Equipment Class Ref: C__ Large jet of MTOW 41k-255k Ib (B737, A320 etc.) 
HeavyAcft AB Heavy jets of MTOW 255k Ib+ 

Large commuter of MTOW 41k-255k lb (small RJs, 
CommuterAcft D ATR42 etc.) 

Medium aircraft of MTOW 12.5k-41k Ib (biz jets, 
MediumAcft E Embraer 120 Learjet 35 etc.) 

Small aircraft of MTOW 12.5k or less (small, single or 
SmallAcft F twin engine Beech90, Cessna Caravan etc.) 
User Class Ref: C = Commercial 
UserClass 1 F = Cargo 
UserClass2 G=GA 
ForeignOD Foreign origin/destination (yes/no) - Ref: domestic 
CeilingHeight Ref: >2500ft 
CeilingHeight<1000ft <1000 
CeilingHeight1001- 
2500ft 1001-2500 
Visibility Ref: 8-10 statute miles (SM) 
Visibility<2SM <2SM 
Visibility2-4SM 2-4 SM 
Visibility4-6SM 4-6 SM 
Visibility6-8SM 6-8 SM 
Crosswind Ref:< 2 knots 
Xwind2-5knts 2-5 knots 
Xwind5-12knts 5-12 knots 
Xwind>12knts al2 
ElectStorm Electrical storm (yes/no) — Ref: no 
IcingConditions Icing conditions (yes/no) — Ref: no 
Snow Snow (yes/no) — Ref: no 
Air Temperature Ref: 15 — 25 deg.C 
Temp<5C <5 deg.C 
Temp5-15C 5-15 deg.C 
Temp>25C > 25 deg.C 
NonhubApt Non-hub airport (yes/no) — Ref: hub airport 
SignificantTerrain Significant terrain (yes/no) — Ref: no 
Notes: 


_Ref: indicates the reference category against which the odds ratios should be interpreted. 
Non-hub airport: airport having less than 0.05% of annual passenger boardings 
Significant terrain: terrain within the plan view of airport exceeds 4,000 feet above the airport 
elevation, or if the terrain within a 6.0 nautical mile radius of the Airport Reference Point rises 


to at least 2,000 feet above the airport elevation. 
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Appendix M provides the results for multivariate logistic 
regression analysis used to obtain the model coefficients 
described earlier. 


Accident Location Models 


Based on the accident/incident data for wreckage loca- 
tions, three sets of complementary cumulative probability 
distribution (CCPD) models were developed in this study. 
With CCPDs, the fraction of accidents involving locations 
exceeding a given distance from the runway end or thresh- 
old can be estimated. When the CCPD is multiplied by the 
frequency of accident occurrence, a complementary cumu- 
lative frequency distribution (CCFD) is obtained. The latter 
quantifies the overall frequency of accidents involving loca- 
tions exceeding a given distance from the runway end or 
threshold. 

The CCPD model structure selected was used by Eddowes 
et al. (2001) and is in the following form: 

For the longitudinal distribution, the basic model is: 


P{Location > x}=e-*" (10) 
where 


P{Location > x} =the probability the overrun/undershoot 
distance along the runway centerline 
beyond the threshold is greater than x; 
x =a given location or distance beyond the 
threshold; and 
a, n= regression coefficients. 


For the transverse distribution, the same model structure 
was selected. However, given the accidents transverse loca- 
tion is not reported, in general, if the wreckage location is 
within the extended runway lateral limits, it was necessary to 


use weight factors to reduce model bias, particularly for mod- 
eling the tail of the probability distribution. Therefore the 
model can be represented by the following equation: 


P{Location > y} = e-»”" (11) 
where 


P{Location>y} = the probability the overrun/undershoot 
distance from the runway centerline is 
greater than y (P{Location<=0} = c); 
y=a given location or distance beyond the 
threshold; and 


b, m= regression coefficients. 


The correlations between the overrun and undershoot dis- 
tances to the lateral distance relative to the runway axis also 
were evaluated for assessing the correlation between x and y 
locations. A high correlation would suggest the best geome- 
try for RSAs is not a rectangle. 

When plotting the percent of accidents beyond a certain 
distance from the threshold, shown in Figure 18, it can be 
noted that an RSA with 1000 ft in length will encompass close 
to 95 percent of all landing overruns. It should be noted that 
the raw data includes reported accidents and incidents, but 
incidents were weighted to account for unreported cases. 

Figure 19 depicts the distribution of raw lateral distances 
from the extended runway centerline. For many events the 
distance was very close to the runway centerline and the 
actual distance was not reported. For such cases when possi- 
ble, the y-distance was assumed to be 0.0. As mentioned 
earlier, weighting factors were used to obtain unbiased esti- 
mates at the tails of the distribution. In this case, weighting 
was applied to the events having y-distances above 400 ft. 

The LDOR CCPD for normalized distances is shown in 
Figures 20 and 21. Using transformed distances, a 1000 ft-long 
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Figure 18. LDOR location model using raw (nonnormalized) 


distances. 
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Raw Lateral Distances Model for LDOR 
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Figure 19. LDOR lateral location model using raw 
(nonnormalized) distances. 


Normalized Distances Model for Landing Overruns 
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Figure 20. LDOR location model using normalized distances. 
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Figure 21. LDOR lateral location model using normalized 
distances. 
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RSA will encompass approximately 80 percent of all landing 
overruns. 

The probability that the point of first impact is beyond a 
certain distance for landing undershoots is depicted in 
Figures 22 and 23, for raw distances, and in Figures 24 and 25, 
for normalized distances. For nearly 13 percent of landing 
undershoots, the aircraft point of first impact will occur at 
distances greater than 1000 ft from the runway threshold. 

The raw location probability trend for takeoff overruns is 
depicted in Figures 26 and 27. From the raw, unweighted 
accident and incident data, close to 20 percent of takeoff 
overruns will occur beyond a 1000 ft distance from the 
threshold. The normalized distance models for takeoff over- 
runs is presented in Figures 28 and 29. 

For each set of location models, one model was developed 
with the raw distance locations and one model used normal- 
ized distances relative to terrain type, runway elevation, and 
the air temperature during the accident/incident. Tables 10 
and 11 show the location models developed in this study. 

The sample sizes available to develop the models shown in 
Table 11 were smaller than those used for the models shown 
in Table 10. A number of investigation reports provide only 
the distance from the threshold, but not the lateral distance. 
Sample sizes for normalized models also are smaller than 
those developed with raw data. For a few cases in each acci- 
dent group there was no information on the terrain type used 
to normalize the distance. 


Analysis of RSA Geometry 


The correlation between the overrun and undershoot dis- 
tances to the lateral distance relative to the runway axis was 
evaluated to define the geometry of the safety areas. RSA are 
normally rectangular-shaped areas, but it was possible that a 
strong correlation between longitudinal and lateral could 


exist. In other words, a statistical analysis was necessary to 
evaluate if greater longitudinal distances for wreckage loca- 
tion can lead to greater transverse distances. 

The correlation between the longitudinal and lateral dis- 
tance for each type of event is shown in Table 12. 

Although the correlation between x and y locations is not 
zero for LDOR and LDUS (P < 0.05), the level is relatively 
low; it was assumed that the correlation is not important. 
This leads to the assumption that the transverse location dis- 
tribution of accidents is fairly constant along the longitudinal 
locations from the threshold. 


Consequences 


As described earlier, accident costs were used to integrate 
consequences related to injuries and property loss into a single 
parameter. The initial intent was to relate the consequences, 
represented by the accident cost, with the wreckage distance for 
the accident. The relationship could be used to estimate the 
consequences of accidents based on the wreckage location, 
providing a link between the location and consequences mod- 
els. Unfortunately these relationships were found to be quite 
poor, as consequences depend not only on the speed when the 
aircraft departs the runway, but also the nature and location of 
existing obstacles, as well as the type and size of aircraft. 

Additional analysis attempted to relate accident location 
with aircraft damage. Four categories of damage—none, 
minor, substantial, and hull loss—were correlated to accident 
location. The use of raw distances proved to hold very low 
correlations between wreckage path distance and the aircraft 
damage. However, there was an improvement when normal- 
ized distances relative to terrain, elevation, and temperature 
were used. The correlations are quite reasonable, as shown in 
Table 13. 
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Figure 22. LDUS location model using raw (nonnormalized) 


distances. 
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Figure 23. LDUS lateral location model using raw 
(non-normalized) distances. 
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Figure 24, LDUS location model using normalized distances. 


Normalized Lateral Distances Model for LDUS 
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Figure 25. LDUS lateral location model using normalized 
distances. 
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Raw Distances Model for Takeoff Overruns 
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Figure 26. TOOR location model using raw (nonnormalized) 
distances. 
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Figure 27. TOOR lateral location model using raw 
(nonnormalized) distances. 
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Figure 28. TOOR location model using normalized distances. 
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Normalized Lateral Distances Model for TOOR 
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Figure 29. TOOR lateral location model using normalized 


distances. 


Both Spearman R and Kendal Tau correlation coefficients 
provide an indicator of the degree of co-variation in the vari- 
ables. Both tests require that variables are represented at least 
in ordinal scale (rank), which is the case for aircraft damage. 
While Spearman R has an approach similar to the regular 
Pearson product-moment correlation coefficient, Kendall 
Tau rather represents a probability. 

Despite these reasonable correlations, a more rational 
approach to model consequences was preferred to assess the 
effect of different obstacles at various locations in the vicinity 
of the RSA. Examples of such obstacles include fences, drops 
and elevations in the terrain, existing facilities, culverts, ALS, 
and ILS structures, trees, etc. 


Modeling Approach 


The main purpose for modeling consequences of aircraft 
accidents is to quantify the risk based on the probability of 
occurrence and the results in term of injuries and property 
loss. It was not possible to develop one model for each type of 


accident, as previously done to model frequency and location. 
However, a rational probabilistic approach is suggested to 
evaluate the probability of accidents or serious accidents. 

The basic idea is to use the location model to estimate the 
incident occurrences when the aircraft will have high energy 
resulting in serious consequences. Figure 30 can be used to 
illustrate and help understand this approach. 

The x-axis represents the longitudinal location of the 
wreckage relative to the threshold. The y-axis is the probabil- 
ity that the wreckage location exceeds a given distance “x.” 
The location distance can be normalized or not, according to 
the criteria selected. 

In this example, an obstacle is located at a distance “D” 
from the threshold and the example scenario being analyzed 
is an aircraft landing overrun incident. The figure shows an 
exponential location model developed for the specific acci- 
dent scenario, in this case, landing overrun. 

There are three distinct regions in this plot. The first region 
(medium shaded area) represents those occurrences that the 
aircraft departed the runway, but the exit speed was relatively 


Table 10. Summary of X-location models. 


en 

Type of Type of Model Eq.# R # of 

Accident Data Points 
LDOR Raw Pld sxj= 7-0-00387 1x9999179 (12) 99.8% 257 
Normalized Pld >x} = 7 0.004692.x98%513 (13) 99.5% 232 
LDUS Raw Fidsxj~ 7 0.024445. 6232 (14) 98.52% 82 
Normalized {d > x} = 0702022078198 (15) 99.1% 69 
TOOR Raw Pld > x} = 070200103319 (16) 99.0% 76 
Normalized A(d>a)< 070-0033 64.x 9807138 (17) 98.5 72 


where P{d > x} is the probability the wreckage location exceeds distance x from the threshold, 
and x is the longitudinal distance from the threshold. 
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Table 11. Summary of Y-location models. 


a SE 
Type of Type of Model Eq.# R # of 
Accident Data Points 
LDOR Raw P{d>y}= 67-0-20174 0489009 (18) 94.7% 141 
Normalized Pld > y} =e70-243692 Teed (19) 93.4% 138 
LDUS Raw Pld> y)= 0.409268 9991851 (20) 92.0% 48 
Normalized Pld> yy ie 0-19539 9" (21) 90.3% 41 
TOOR Raw Pld> y)= 9-0-182098 y9 48546 (22) 95.6% 44 
Normalized P{d> y) = 70181046 yA (23) 97.1% 42 


low, and the aircraft came to a stop before reaching the exist- 
ing obstacle. The consequences for such incidents are expected 
to be none to minor as the aircraft may hit only frangible ob- 
jects (e.g., threshold lights) within these small distances. 

The rest of the curve represents events that the aircraft 
exited the runway at speeds high enough for the wreckage 
path to extend beyond an existing obstacle. However, a por- 
tion of these accidents will have relatively higher energy and 
should result in more severe consequences, while for some 
cases the aircraft will be slow when hitting the obstacle so that 
catastrophic consequences are less likely to happen. 

Using this approach, it is possible to assign three scenarios: 
the probability that the aircraft will not hit the obstacle (re- 
sulting in none or minor consequences); the probability that 
the aircraft will hit the obstacle with low speed and energy 
(with substantial damage to aircraft but minor injuries); and 
the probability that the aircraft will hit the obstacle with high 
energy (with substantial damage and injuries). 

For events with low energy when impacting the obstacle, it 
is possible to assume that if no obstacle was present the aircraft 
would stop within a distance A from the location of the obsta- 
cle. The problem is to evaluate the rate of these accidents hav- 
ing low speeds at the obstacle location and this is possible based 
on the same location model. This probability can be estimated 
by excluding the cases when the speed is high and the final 
wreckage location is significantly beyond the obstacle location. 

A similar approach was developed to combine the longitu- 
dinal and transverse location distribution with the presence, 
type, and dimensions of existing obstacles. The basic approach 
is represented in Figure 31 for a single and simple obstacle. 

A few simplifying assumptions were necessary when devel- 
oping this approach. One simplification is to assume the 
lateral distribution is random and does not depend on the 


Table 12. Correlation between lateral and 
longitudinal overrun/undershoot distances. 


Type of R RX CI 95% p n 
Event 
LDOR 0.320 10.2% 0.20 - 0.43 < 0.0001 235 
LDUS 0.316 10.0% 0.11 - 0.50 0.0040 81 


TOOR 0.113 1.3% -0.12 a 0.33 0.3430 73 


presence of obstacles. This is a conservative assumption be- 
cause there are events when the pilot will avoid some obsta- 
cles if he has some control of the aircraft. The database 
contains a number of cases when the pilot avoided ILS and 
Approach Lighting System (ALS) structures in the RSA. 

Asecond assumption is that the aircraft follows a path near 
parallel to the extended runway axis. Again, this assumption 
will lead to calculations of higher than actual risk and is con- 
servative. The aircraft may hit or avoid obstacles in paths that 
are nonparallel to the runway axis. 

The shaded area in Figure 31 represents the area of analy- 
sis. Accident data was considered relevant when wreckage 
location challenged an area of 2000 x 2000 ft beyond the 
threshold. The example shown in the figure depicts an over- 
run example. 

Obstacle 1 is located at a distance x,, y, from the threshold 
and has dimensions W, x L;. When evaluating the possibility 
of severe consequences it is possible to assume this will be the 
case if the aircraft fuselage or a section of the wing close to 
the fuselage hits the obstacle. Thus, it is possible to assume 
the accident will have severe consequences if the y location 
is between Y, and Y, as shown in the figure. Based on Equa- 
tion 11 for transverse distance, the probability the aircraft axis 
is within this range can be calculated as follows: 


m —by"? 
en bye -—e sf 
Pe = a (24) 


where 


P,,= the probability of high consequences; 
b, m= regression coefficients for y-location model; 
Y,= the critical aircraft location, relative to the obstacle, 
closest to the extended runway axis; and 


Table 13. Correlation between normalized 
wreckage location and aircraft damage. 


Type of Event Sample Size Spearman R___ Kendall Tau 
LDOR 224 0.62 0.49 
LDUS 68 0.30 0.23 
TOOR 67 0.55 0.44 
All 359 0.56 0.44 


xo) 
~ 
Ea 
a3 
§ ® Prob of Incident with d < D, 
2 Ki P{d < D,} = 100% - P{d > D,} 
Prob of Incident with d > D, 
P{d > D,} low energy 
Pi.= P{d > D,}- P{d > D,+A} 
P{d > D,+A} Prob of Incident with d > D, +A 


high energy 
Prign =P{d > D+A} 


D,  D,tA 


Distance x from threshold 
Do is distance to Obstacle, d is distance the aircraft'came to stop 


Area (lightly shaded) between Dy and Dy +A represent % occurrences 
at low speed (energy) when hitting obstacle (low consequences) 


Figure 30. Approach to model consequences 
of overrun/undershoot accidents. 


Y;= the critical aircraft location, relative to the obstacle, 
farther from the extended runway axis. 


The same example is depicted in Figure 32 showing the 
probability of severe consequences can be represented by the 
lightly shaded area in the probability distribution. 

Combining this approach with the longitudinal distribu- 
tion approach and the possibility of multiple obstacles, the 
risk for accidents with severe consequences can be estimated 
using the following model: 


_pyltt 
is (a — by 
{a ~ ———— en xis Ai)" (25) 


i=] 
where 


N= the number of existing obstacles; 
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a, n= regression coefficients for the x-model; and 
A;= the location parameter for obstacle 7. 


The value of A may be estimated based on Kirkland’s model 
for aircraft deceleration over different types of terrain (Kirkland 
et al., 2004) and crashworthiness speed criteria for aircraft. It 
should be noted that A depends on the type of terrain, type and 
size of aircraft, and type of obstacle. Frangible objects in the RSA 
are less prone to causing severe consequences. Lighter aircraft 
may stop faster and the landing gear configuration also may 
have an effect on the aircraft deceleration in soft terrain, but 
these factors are not accounted for in Kirkland’s model. 

The probability and location models should provide a 
quantitative assessment based on operating conditions for a 
specific airplane landing or takeoff at a specific runway. The 
consequences model should provide a qualitative assessment 
of the severity of an accident, based on the location model 
and the existing runway characteristics, to include dimen- 
sions of existing RSA, airplane weight, type, location and size 
of obstacles, and the topography of the surrounding terrain. 

The procedure will allow modeling overrun and under- 
shoot risks for the conditions of the airport being evaluated. 
The probability of the accident occurring, as well as stopping 
location distances, will be compared to existing geometry of 
safety areas and existing obstacles to assess the possible con- 
sequences of the accident, at least qualitatively. 


Cost of Accidents 


As described in the previous chapter, the direct costs of ac- 
cidents and incidents were estimated for each event having 
sufficient information for the computation. This section 
presents a summary of these costs. 


Figure 31. Modeling consequences. 
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2000ft 


38 


Lateral Location 
Probability 


Figure 32. Modeling likelihood of striking an obstacle. 


Accident and Incident Average Direct Costs 
(2007 dollars) 


Average Cost ($ Million) 
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Figure 33. Direct cost of accidents and incidents. 


As mentioned earlier, the intent was to use the data and 
find relationships between certain parameters of the accident 
(e.g., wreckage path distance) and the number and level of 
injuries, as well as damage to aircraft. The total consequences 
were estimated in terms of total direct costs for injuries, air- 
craft damage, and accident investigation. 

Figure 33 depicts the average cost by type of accident and by 
severity. Most of the cost for LDORs is attributed to loss of 
property or aircraft damage. On the other hand, loss of dollars 
due to injuries is significantly higher for LDUSs, most likely 
due to the high speed and energy during these accidents. 

The average loss of property among the three types of ac- 
cidents was fairly similar. As expected, the cost of incidents 
was significantly lower for all three types of events. The cost 
of investigation is not represented in the figure, but rather is 
shown in Table 14. 


Table 14. Total and investigation costs 
(2007 dollars, millions). 


Type of Event Total Cost _ Investigation Cost 
LDOR ACC 7.08 0.17 
LDOR INC 0.11 0.00 
LDUS ACC 26.75 0.45 
LDUS INC 0.06 0.00 
TOOR ACC 22.60 0.46 
TOOR INC 0,26 0.00 


The costs for injuries, aircraft damage, and accident/incident 
investigation are available in the accident database for each 
event included in this study. Appendix L provides more details 
on the calculation of accident costs. 


CHAPTER 4 
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Practical Application of Models 


Upon developing the risk models, it is necessary to inte- 
grate them to allow risk assessment of RSAs under specific 
conditions. A practical application of such models for exam- 
ining a specific RSA will involve estimating the risk that an 
aircraft operating under specific airport conditions will chal- 
lenge the existing RSA and stop beyond the available RSA 
limits or crash into existing obstacles. 

This chapter includes a step-by-step procedure to evaluate 
an RSA using the models and the approach developed in this 
study. The example presented is for a hypothetical Runway 
07/25, but actual NOD for an existing airport was collected 
from 2002 to 2004 and is used to illustrate the practical 
application of the approach developed. The procedure repre- 
sented in Figure 34 will allow assessing the probability that an 
incident will occur during the aircraft operation and that as a 
result, the consequences are likely to be severe. The initial 
process will involve the following steps: 

The initial process will involve the following steps: 


1. Select the specific RSA to analyze and gather information 
to include dimensions, type of terrain, and existing obsta- 
cles adjacent to the RSA. 

2. Collect (or estimate) a representative traffic sample for the 
three crash scenarios (LDOR, LDUS, and TOOR) chal- 
lenging the RSA being evaluated. 

3. Divide the RSA into sections comprising specific crash 
scenarios. Each scenario should include the distance from 
the threshold and two lateral distances relative to the ex- 
tended runway axis. 

4. Using the frequency models and location models, estimate 
for every operation the probability for each crash scenario 
and type of operation. Based on these data, develop the 
frequency distribution for each crash scenario and for 
each type of incident (LDOR, LDUS, and TOOR). 

5. For each type of operation, determine the percentage of oc- 
currences having a risk higher than a selected level or TLS. 


6. Determine the weighted frequency distribution for all 
types of incidents together. 

7. Repeat the analysis for the remaining runway ends of the 
airport. 

8. Account for the risk exposure and estimate the probabil- 
ity, in terms of accidents per year, for each type of incident 
and for each RSA of the airport. 

9. Classify the RSA according to the percentage of high-risk 
operations. 


Step 1—RSA Details 


The RSA to be evaluated should be characterized by type of 
terrain, dimensions, and type, size, and location of obstacles. 
The analysis is for the approach end of Runway 25 and is a 
simple example because the obstacles that can cause severe 
consequences are well defined; the RSA is rectangular and 
symmetric, and it is surrounded by a body of water, as de- 
picted in Figure 35. 


Step 2—Collect Representative 
Traffic Sample 


This step will involve gathering normal operations data for 
the airport. A second alternative for this step is to estimate the 
expected operations based on the planned flight schedule for 
the facility. When using historical data, two possible sources 
are: the Aviation System Performance Metrics (ASPM) data, 
which identify the takeoff and landing runways in hourly seg- 
ments, the runway, and direction for each flight; and the 
Enhanced Traffic Management System Counts (ETMSC), 
which provides information on traffic counts by airport or by 
city pair for various data groupings. These same sources were 
used to obtain NOD for this study. The sample flight data are 
for operations that may challenge the RSA being evaluated in 
the event of an aircraft overrun or undershoot. 
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Characterize Rwy End to 
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3 — Define Crash 
Scenarios and Related 
Distances 


4 - Estimate Risk 


Figure 34. RSA risk assessment. 


The information on flights should be coupled with weather 
conditions obtained from the airport weather station. The rep- 
resentative sample (e.g., 1 year) should contain the informa- 
tion on the parameters used in the frequency models (Table 7). 

Aircraft operations challenging the RSA at the runway end 
shown in Figure 35 are those arriving on Runway 25 (LDUSs) 
or arriving and departing from Runway 07 (LDORs and 
TOORs). To run the analysis, a sample of the traffic landing 
and taking off on Runway 07 and landing on Runway 25 is 
necessary. The sample must be representative of the existing 
or planned conditions for Runway 07/25. Preferably, the 
sample should cover at least 1 year of operations, but it may 
not be necessary to obtain data for each month. A sample 
comprising at least 4 months of operations, but characteriz- 
ing the whole year should suffice (e.g., January, April, July, 
and October), such that different seasons, environmental 
conditions, and seasonal variations of traffic are represented. 


5 — Characterize Risk 
Frequency Distribution 


9 - Classify Risk, Compare 
RSAs, Prioritize Mitigation 
Actions 


ire eda 


7 - Determine Total % of 
Ops at Risk Above TLS 


Between 2002 and 2004, a sample of historical data from an 
existing airport was gathered for this example. Data include 
landing operations for Runway 25, landing operations for 
Runway 07, and takeoff operations for Runway 07. In the 
event of an incident, these flights may challenge the RSA for 
this example. 


Step 3—Define Crash Scenarios 


This step still requires some judgment from the analyst. In 
the example shown in Figure 35, the RSA adjacent to the Run- 
way 25 approach end is only 100 ft long by 500 ft wide. 

When applying the risk assessment procedure to a partic- 
ular RSA, it is important to recognize that not every RSA has 
a standard or rectangular shape. In many cases, a combina- 
tion of the shapes and sizes as well as obstacles may exist. 
Other factors, such as the type of obstacle, will often need a 


Figure 35. RSA on approach end of Runway 25. 


subjective assessment and the use of some simplification to 
run the risk analysis. 

For the RSA configuration in the example, there are only 
four crash scenarios that may lead the aircraft to fall into the 
water if overrunning or undershooting the runway: 

Crash Scenario 1. During a landing or takeoff operation on 
Runway 07, the aircraft will overrun the RSA within its 500 ft 
wide boundaries and fall after running 100 ft. 

Crash Scenario 2. During a landing on Runway 25, the air- 
craft will undershoot the RSA within 500 ft wide boundaries 
and before the RSA end, 100 ft from the threshold. 

Crash Scenario 3. Same as Crash Scenario 1, but the aircraft 
will fall into the water outside the 500 ft wide boundaries of 
the RSA. 

Crash Scenario 4. Same as Crash Scenario 3, but the aircraft 
will undershoot before the runway end and outside the 500 ft 
wide boundaries of the RSA. 

It should be noted that the small number of crash scenar- 
ios identified is for this simple example only. The presence of 
obstacles or the asymmetry of the RSA would lead to addi- 
tional possible crash scenarios. For example, if the RSA had 
an ILS structure, two additional crash scenarios should be 
considered: 

Crash Scenario 5. During a landing or takeoff operation on 
Runway 07, the aircraft will overrun the RSA within the cen- 
ter area of the RSA and hit the ILS structure with enough 
energy for severe consequences. 

Crash Scenario 6. Same as Crash Scenario 5, but the aircraft 
will undershoot in the RSA, before the ILS structure and 
strike it. 

In this case, Crash Scenario 1 should be modified so that 
the aircraft would overrun or undershoot the runways 
between the RSA 500ft boundaries in a path that avoids the 
ILS structure and the aircraft falls into the water. No modi- 
fication would be required for Crash Scenario 2, when the 
aircraft undershoots the runway before the beginning of 
the RSA. 


Step 4—Estimate the Risk 


For each operation in the NOD sample, it is necessary to 
calculate the risk. An example calculation for one landing op- 
eration will be described to help understanding of the 
process. 

Suppose one of the landing operations on Runway 07 is 
characterized by the parameters depicted in Table 15. 

The information is used in Equation 7 to compute 
the probability the aircraft will overrun Runway 07 during 
a landing operation under these conditions, thus challeng- 
ing the RSA being evaluated. The computation is the 
following: 
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Table 15. Example normal operation 
on Runway 07, hypothetical airport. 


Parameter Value Note 
Aircraft B738 Boeing 737-800 
Equipment Class N Large jet 
User Class Cc Commercial 
Equipment Type J Jet 
Wind Direction 340 
Wind Speed Knts 13 
Ceiling ft 500 
Visibility Sm 1.0 
Temperature C 30 
Fog 1 Fog present 
Icing 0 No icing 
Elec. Storm 0 No elect. Storm 
Frozen Precipitation 0 No frozen precip. 
Snow 0 No snow 
XWind Knts 13 calculated 
Rain 0 No rain 
Foreign Origin/Dest. 0 Domestic 
Significant Terrain 0 No significant terrain 


b =—15.456 + 0.551(0) —2.113(0) —1.064(0) —0.876(0) 
+0.445(0)—0.857(0)+1.832(1) + 1.639(0) + 2.428(1) 
+1.186(0) +1.741(0) +0.322(0)—0.532(0)+1.566(0) 
+1.518(1) + 0.986(0) +1.926(0) +1.499(0) —1.009(0) 
~0.631(0) + 0.265(1)+1.006(0) +0.924(0) =—9.413 


The probability that an aircraft will not be able to stop dur- 
ing a landing operation and overrun the runway under these 
conditions is therefore computed as follows: 


P{LDOR} = 1 g165x104 


As expected, the risk of an incident is high for such unfa- 
vorable conditions: strong cross wind, low visibility, and low 
ceiling. After calculating the likelihood of an overrun, the 
next step in the process is to estimate probability the aircraft 
will stop beyond the RSA or hit an existing obstacle. 

The area surrounding the RSA is a body of water. In this 
case, it is possible to assume the aircraft will be lost if it over- 
runs the RSA and falls into the water. Distance x from the 
runway end is simply the length of RSA measuring 100 ft, and 
this is the value to be entered in the normalized x-location 
model given by Equation 13. It should be noted that the 
parameter “A” from general Equation 25 in this case should 
be set to 0 because the aircraft will fall into the water ifits stop 
location simply exceeds 100 ft beyond the threshold. 

The calculation is given by: 


P{d>x}= e-0.004692x9-824513 
P{d>x}= e70.004692x1000-824513 _ gy a4, 
The probability an aircraft will end in the water if over- 


running the runway during landing is high because the RSA 
is very short. 
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As previously described, it is assumed the aircraft will over- 
run in a path that is nearly parallel to the runway axis. The 
probability that the aircraft will overrun the RSA under Crash 
Scenario 1 is given by a product of three probabilities: the 
probability of occurring the event (overrun); the probability 
the aircraft will stop beyond 100 ft from the threshold; and 
the probability the aircraft will remain within the RSA lateral 
limits during the overrun. Mathematically: 


Prob{CS 1} = Prob{LDOR) x Prob{x > 100ft} 
x (1-Prob{|y| > 250 ft) 


And Prob{|y| > 250 ft} is that given by: 
P{d> y} = e70.20174x2500-489009 _ 5 Qos 


Probability for crash scenario 1 is: 8.165 x 10% x 0.81 x 
(1-0.050) = 6.28 x 10° 

Similarly, the probability for crash scenario 2 is simply 
given by: 


Prob{CS 3) = Prob{LDOR) x Prob{|y| > 250 ft 


In this situation the aircraft only needs to overrun the run- 
way by a small margin; as long as the transverse deviation is 
greater than 250 ft, the aircraft will end up in the water. The 
probability for crash scenario 3 is calculated as follows: 


P{CS 3} = 8.165 x 10> x 0.050 = 4.083 x 10° 


The last step is to compute the total probability the aircraft 
will overrun the runway during landing and fall into the water 
with severe consequences: 


P{LDOR Severe} = P{CS 1} + P{CS 3} = 6.28 x 10° 
+ 4.08 x 10°°= 6.69 x 10° 


According to the FAA criteria described in Table B1-2 from 
Attachment 1 to Appendix B, the event is probable and such 
probability is unacceptable, as shown in the FAA Risk Matrix 
depicted in the same Attachment (Figure B1-1), even when 


Frequency 


Risk threshold (e.g. 1x10”) 


there are major consequences. Considering the criteria set by 
FAA, the risk is considered high and suggests the operation is 
not safe enough under such conditions. However, the risk 
estimated for such conditions at any airport should be con- 
sidered for planning and risk mitigation strategies only as it 
represents an “average” risk level for such conditions. 


Step 5—Characterize Risk 
Frequency Distribution 


The same procedure described for Step 3 can be used to 
compute the probability of severe consequences for every 
landing operation for Runway 07 that is part of the NOD sam- 
ple. Each operation has a different risk associated with it. If all 
these risks are estimated, it is possible to build a histogram 
depicting the distribution of risk, as illustrated in Figure 36. 


Step 6—Determine Percentage of 
Operations with Risk Above TLS 


An example of the probability distribution generated by 
the prototype software developed under this project is shown 
in Figure 37. Each bar represents the percentage of operations 
having a specific risk level. The line represents the percentage 
of operations having risk higher then the level selected. In this 
example, if a TLS of 1:10000000 is selected, approximately 
9 percent of the operations will be subject to undesirable levels 
of risk. This is useful, as it evaluates the percentage of opera- 
tions with risk above a selected TLS. The area in dark bars 
represents such flights. 

The same process is used to estimate the percentage of 
operations having a risk level above the selected TLS for 
LDUSs on Runway 25 and for TOORs on Runway 07, using 
the models associated with these types of events. The proba- 
bility distribution of risks then can be characterized for each 
type of accident in the vicinity of that RSA. 


Frequency distribution estimated from 
expected traffic conditions at the 
airport/threshold and risk models 


Region of Normal 
Operations and High Risk 


Probability WL > X 


X is the RSA length or distance to existing obstacle 


Figure 36. Frequency distribution of risk. 
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LDOR Probability Distribution of Risk 
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Figure 37. Typical frequency distribution of risk 


from prototype software. 


Step 7—Total Percentage of 
Operations with Risk Above TLS 


The three probability distributions (LDOR RWY 07, LDUS 
RWY 25, and TOOR RWY 07) can be combined to provide 
an overall probability distribution that a serious accident will 
occur in the specific RSA. The percentages for each cell 
should be weighted relative to the traffic for each type of 
operation. A hypothetical example is presented to help un- 
derstanding of the process. 

Assuming these are the conditions for the airport being 
evaluated if the TLS is set to 1 x 10~, results for the approach 
end of Runway 25 are summarized on Table 16. 

There are significant differences to risk levels for each op- 
eration, particularly noting that 34 percent of landings on 
Runway 07 have risk above the threshold level selected for 
this analysis. But there also are notable discrepancies in terms 
of exposure to visibility, ceiling height, and fog between land- 
ings on Runway 07 and those on Runway 25. 

Over 30 percent of landings on Runway 07 take place in 
visibility under 2 statute miles, compared to 1.67 percent on 
Runway 25. In a related measure, almost 40 percent of land- 
ings on Runway 07 experienced fog versus less than 7 percent 
for Runway 25. Moreover, 39 percent of landings on Runway 
07 took place in ceiling height under 1000 ft, while the equiv- 
alent for Runway 25 is only 3.9 percent. 


These differences are most likely related to the airport’s 
runway use policy. In fact, data on the airport’s runway usage 
patterns revealed that landings on Runway 07 are relatively 
rare. It is reasonable to assume this runway is used for land- 
ings only for exceptional circumstances, such as adverse wind 
conditions, and this also would explain the discrepancy in 
risk exposure for landings on Runway 07 and Runway 25. 

Using these numbers, the percentage operations with risk 
above the TLS and challenging the RSA being evaluated can 
be estimated as follows: 


(0.073 x 0.34 + 0.09 x 0.01 + 0.3 x 0.02) 
(0.073 + 0.09 + 0.3) 


% > TLS = = 6.9% 

Based on the annual number of operations at the airport, 
the analysis also can be used to estimate the annual rate of 
accidents or the number of years likely to take for a severe 
accident to occur. These are useful parameters as they allow 
comparing different RSAs of the same or different airports to 
identify and prioritize the RSAs requiring risk management 
actions or improvements. 


Steps 8 and 9—Repeat the Analysis 
for Other Runway Ends 


In repeating the analysis for the remaining runway ends of 
the same airport, it is possible to evaluate which RSA poses 


Table 16. Summary results for hypothetical airport. 


Operation % of Movements Ops @ High Sample Size 
Risk 
Landing on RWY 07 71.3% 34% 487 
Takeoff on RWY 07 9.0% 1% 606 
Landing on RWY 25 30.0% 2% 2020 
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Table 17. Summary risk for hypothetical airport. 


Annual Ops Annual Ops w/ Risk 
Roe Challenging RSA __#OPS>TLS Above TLS 
Approach End RWY 11 300,000 6% 18,000 
Approach End RWY 29 160,000 1% 11,200 
Approach End RWY 03 410,000 1% 4,100 
Approach End RWY 21 40,000 14% 5,600 


higher risk during the year. The assessment may help improve 
runway use so that risk is minimized for the airport. To com- 
pare different RSAs, it is preferred to use a parameter that can 
provide a direct comparison. One possibility is to transform the 
percentage of operations above TLS into a volume of annual 
operations above TLS by multiplying the percentage by the vol- 
ume of flights challenging the specific runway end. Using such 
a parameter, it is possible to compare multiple RSAs of the same 
airport, or even for different facilities, to determine and priori- 
tize which RSAs may require risk mitigation actions. 


Using another hypothetical example, the following param- 
eters can characterize the conditions for an existing airport 
having two runways: 11/29, and 03/21. 

In Table 17, the approach end for Runway 11 has a higher 
number of annual operations with risk above the selected 
TLS. A number of risk mitigation measures may be priori- 
tized to improve safety for such area. Some safety manage- 
ment alternatives may include modifying runway use for the 
airport, installing new NAVAIDS, or improving the RSA, 
among other procedures. 


CHAPTER 5 
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Conclusions and Recommendations 


Current standards for RSAs are fairly rigid as they depend 
only on the type and size of aircraft using the runway. How- 
ever, numerous factors affect the operations that may lead to 
aircraft overruns and undershoots. In reality, operations are 
carried out under varying levels of safety. 

This study introduces a more comprehensive approach to 
evaluate the degree of protection offered by a specific RSA 
and provides a risk-based assessment procedure that is ra- 
tional and accounts for the variability of several risk factors 
associated with aircraft overruns and undershoots. In addi- 
tion this study provides risk models that are based on com- 
prehensive evidence gathered from aircraft accidents and 
incidents in the United States and other countries. Informa- 
tion gathered from these events has been organized into a 
database that may be used for future studies on airport risk 
assessment. Moreover, the basic approach utilized in this 
research can be used to model other types of accidents 
occurring near airports, particularly for veer-off and takeoff 
and crash accidents. 

Existing and new techniques were integrated into a practi- 
cal approach that uses NOD and normalization procedures. 
Incorporating NOD allows the user to account for different 
aircraft flight risk exposure to assess the criticality of risk 
factors. Although with some limitations, the normalization 
procedures increased the pool of comparable data for model 
development. 

Despite some limitations on availability of information to 
develop the models presented in this report, the models 
provide useful information and are integrated in a practical 
approach that will be useful for RSA planning and airport 
safety management. Prototype software also was developed as 
part of this effort and serves to verify how the several models 
interact and determine risk probability distributions for an 
RSA subject to specific conditions. 


Major Achievements 


Development of Accident 
and Incident Database 


One of the accomplishments of this study was the devel- 
opment of a comprehensive and organized database of acci- 
dents and incidents. It includes the information gathered for 
events involving aircraft LDORs, LDUSs, and TOORs. The 
database has editing and querying capabilities, and includes 
most parameters associated with the events, including air- 
craft, airport, runway, operation, causal factors, and conse- 
quence information used in the modeling process and can be 
used for future studies on airport safety. The database over- 
came the lack of depth of many accident databases developed 
in previous studies. 

Analysis of the database revealed which anomalies are more 
frequent for each type of incident and the factors contributing 
to these events that would be desirable to incorporate in the 
modeling process. 


Frequent Anomalies in Aircraft Overruns 
and Undershoots 


When possible, the anomalies present during the occur- 
rences were listed for each accident and incident included in the 
database. Most of these anomalies were taken from accident in- 
vestigation report lists of causal and contributing factors; how- 
ever, other anomalies were listed, even when not included in the 
report if they were present during the occurrence. 

These anomalies were summarized by type of event and by 
severity of the event. The summary provided is useful to help 
airport operators understand how these factors may lead to 
accidents, and it helped to identify those factors more fre- 
quently associated with aircraft overrun and undershoot 
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events. Whenever possible, these factors were incorporated in 
the modeling approach. 


Normal Operations Database 


Another achievement in this research is the availability of 
a normal operations database for U.S. flight operations. The 
NOD sample comprising 242,420 flights is a representative 
sample of 95.8 percent of flights in the United States and cov- 
ers the great majority of airport types, geographical regions, 
and operational and meteorological parameters. The devel- 
opment of this database was crucial in quantifying and char- 
acterizing accident risk factors, as well as the development of 
accident frequency models. In addition, the NOD provides an 
assessment of flight risk exposure and may be used for future 
studies on airport and runway safety. 


Development of Risk Models for RSA 


Each set of models includes one model for frequency and 
two models for location: one developed from raw distance 
data and one based on normalized distance data. A rational 
approach was developed to model severe consequences and 
serves for any of the model sets. 

These models incorporate several risk factors identified for 
each type of accident. These include a range of meteorologi- 
cal conditions, operational parameters, and existing features 
and obstacles present in the RSA and its surroundings. In 
addition the approach allows consideration of multiple risk 
factors in a single model that accounts for their joint influ- 
ences on accident likelihood. Consequently, the models 
developed have substantially improved predictive power, 
with enhanced sensitivity and specificity, versus existing reg- 
ulations based on a small number of risk factors. 


Integrated Practical Approach 


This study also brings a new rational and probabilistic ap- 
proach to integrate the frequency and location models for 
evaluation of the likelihood of an accident with severe conse- 
quences. The approach accounts for the variability and risk 
exposure relative to various factors and provides a proba- 
bilistic assessment of risks. 

The output of the risk assessment is a probability distribu- 
tion of risks for an existing RSA subject to specific meteoro- 
logical and operation conditions. As a result, it is possible to 
evaluate quantitatively the overall risk of accidents for a par- 
ticular runway and associated RSA. 


Prototype Software 


Running a risk assessment for a specific RSA using the ap- 
proach developed in this study requires extensive calculations. 


A sample of NOD for the runway must be representative of the 
traffic occurring in the airport during at least one year so that 
it covers different meteorological conditions for the airport. 
In addition, depending on judgment by the analyst, there may 
be multiple crash scenarios and these will require modeling for 
each of these situations. 

To facilitate these computations, software that incor- 
porates models developed in this study was developed. 
The software allows one to enter existing RSA conditions 
and a sample NOD for each type of operation that may 
challenge the RSA to run the analysis for an existing 
obstacle. Appendix O describes the prototype software de- 
veloped for this project and shows some screens, and 
input and output parameters. 


Model Limitations 


Modeling aviation risks always has been a challenge. Acci- 
dent and incident reports often lack quantitative information 
for causal factors of overrun and undershoot accidents. The 
large quantity of causal factors and the scarceness of reliable 
data limits the accuracy of models developed. Modeling con- 
sequences still depend on some judgment from the analyst. In 
this study it was not possible to make more solid recommen- 
dations on how to treat the obstacle/aircraft interaction and 
how much energy is required to cause severe consequences 
during the impact. 

Most likely, airport operators may need expert help to 
make correct use of the models developed in this study, but 
they should be able to use them effectively with the help of a 
risk assessment professional. The modeling approach on its 
own is not enough to allow anyone to perform a risk analysis 
for RSAs. The assessment of risk associated with overrun and 
undershoot events has to be placed in some context, a process 
to be followed by airport operators that will support their risk 
management decision making. 

A review of existing databases has shown that deviation 
data for air operations are generally sparse, particularly for 
reporting of relevant details on incidents (more detailed data 
generally are available for accidents). Collection of NOD was 
a challenge, particularly to counteract the impossibility of 
accessing the flight operation quality assurance (FOQA) data- 
base from the airlines. 

The models and approach developed in this study can be 
helpful for airport planning and safety management RSA 
improvement actions. However, under no circumstances 
should the models developed in this research be used to as- 
sess real-time operational risks. There are several factors not 
accounted for in the models and their accuracy is not ap- 
propriate for decision making during actual operations or 
emergency situations. 


Recommendations for Future Work 
Improved Normal Operations Data 


Collecting normal operations data for parameters not cov- 
ered in this study would greatly help to improve the approach 
developed in this study. Some of the factors, although con- 
sidered important to modeling aircraft overrun and under- 
shoot events, were not available from the NOD sources used 
in this study. Of particular importance are: the runway criti- 
cality, the presence of tailwind and the runway friction dur- 
ing the operation. Information on these factors could not be 
obtained from the ETMSC and ASPM databases. 

Incorporation of these factors would certainly improve the 
predictive performance of the models and enhance the over- 
all accuracy of the risk assessment. Airline flight data recorder 
(FDR) information is especially relevant to obtain informa- 
tion on runway criticality but access to such data will require 
the cooperation of airlines and industry organizations. 

Even if data from FDRs are not available, the runway crit- 
icality factor may be incorporated in the models using the 
basic distances required for operation of each aircraft type. 
Although this improvement was identified in the course of 
this research, time and financial resources constraints did not 
allow the incorporation of such factors in the present models. 


Availability of Information 
for Accidents and Incidents 


Accident investigation and incident reports seldom contain 
the comprehensive information required to improve model- 
ing. Even when there are standard forms and reports, many of 
the existing fields are not filled by the reporter. In addition, 
some important parameters associated with events relevant to 
this study are not available in the standard forms, or the in- 
vestigator is not aware of their importance for modeling risks. 
Of particular interest is the runway distance required for land- 
ing or takeoff. Without such information or the parameters 
(e.g., actual weight at crash) required to compute the distances 
of interest, it becomes difficult to assess the runway criticality. 

Precise wreckage path, location exiting the runway, and 
final wreckage location are rarely reported, even for accidents. 
Details on the type of obstacle, its dimensions and location 
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are seldom reported. Therefore, it is important to develop 
more comprehensive guidance related to the parameters 
required to improve risk modeling in the future. A list of sug- 
gested parameters to report is presented in Appendix N. 

It should be noted that the availability of information on 
accidents and incidents will only be helpful if these parame- 
ters also are available in NOD. 


Development of Comprehensive Software 


The prototype software developed for this project is fairly 
simple and allows risk assessment of RSA using sample NOD 
for different types of operation challenging a specific RSA. 
However, there are limitations, as resources constrained the 
development of more comprehensive software. The risk as- 
sessment is performed for each type of accident separately 
and does not integrate the results to obtain the overall risk. 
Moreover, the analysis system is capable of evaluating one 
simple and rectangular-shaped RSA or one existing obstacle. 
Although the analysis of other shapes and multiple obstacles 
is possible, each comprises one single analysis that will re- 
quire integration to be performed manually. 

Data normalization is still limited to one type of terrain. 
Ideally, the analysis should include multiple sections with 
different terrain conditions and possibly the capability to 
evaluate risks when Engineered Material Arresting Systems 
are available in the RSA. 


Development of Onboard Real-Time Models 


One of the products of this research is a summary of fac- 
tors or anomalies frequently present during aircraft overruns 
and undershoot. Some of these factors may greatly help pilots 
making good decisions about attempts to go around if the 
information and risk processing capabilities are provided in 
real-time. If onboard computers could process real-time in- 
formation, like approach speed, height above threshold, 
touchdown location, wind, and braking capability based on 
actual aircraft deceleration to estimate remaining distance re- 
quired compared to distance available, an effective warning 
system could be provided by aircraft manufacturers to advise 
pilots on the best emergency procedure (e.g., go around) to 
prevent many undesirable events. 
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APPENDICES 


Appendices to the contractor’s final report for ACRP Project 04-01, “Aircraft Overrun and Undershoot Analysis for Run- 
way Safety Areas,” are available on the TRB website at http://trb.org/news/blurb_details.asp?id=8928. The appendices are the 
following: 


Appendix A: Understanding Aircraft Overruns and Undershoots 
Appendix B: Functional Hazard Analysis 

Appendix C: Key Accident/Incident Database Sources 

Appendix D: Analysis of Unreported Incidents 

Appendix E: List of Accidents for Model Development 
Appendix F: Database Rules 

Appendix G: Normal Operations Data 

Appendix H: Sampled Airports 

Appendix I: Stratified Sampling Strata and Weights 

Appendix J: Calculation of Relevant Terminal Area Forecast Traffic 
Appendix K: Normalization Procedures 

Appendix L: Direct Costs of Accidents 

Appendix M: Results for Multivariate Logistic Regression 
Appendix N: Recommended Reported Information 

Appendix O: Prototype Software for RSA Risk Assessment 


Abbreviations and acronyms used without definitions in TRB publications: 


AAAE American Association of Airport Executives 
AASHO American Association of State Highway Officials 
AASHTO American Association of State Highway and Transportation Officials 
ACI-NA Airports Council International—North America 
ACRP Airport Cooperative Research Program 
ADA Americans with Disabilities Act 
APTA American Public Transportation Association 
ASCE American Society of Civil Engineers 
ASME American Society of Mechanical Engineers 
ASTM American Society for Testing and Materials 
ATA Air Transport Association 
ATA American Trucking Associations 
CTAA Community Transportation Association of America 
CTBSSP Commercial Truck and Bus Safety Synthesis Program 
Department of Homeland Security 
Department of Energy 
Environmental Protection Agency 
Federal Aviation Administration 
Federal Highway Administration 
Federal Motor Carrier Safety Administration 
Federal Railroad Administration 
Federal Transit Administration 
Institute of Electrical and Electronics Engineers 
Intermodal Surface Transportation Efficiency Act of 1991 
Institute of Transportation Engineers 
National Aeronautics and Space Administration 
National Association of State Aviation Officials 
National Cooperative Freight Research Program 
National Cooperative Highway Research Program 
National Highway Traffic Safety Administration 
National Transportation Safety Board 
Society of Automotive Engineers 
SAFETEA-LU Safe, Accountable, Flexible, Efficient Transportation Equity Act: 
A Legacy for Users (2005) 
TCRP Transit Cooperative Research Program 
TEA-21 Transportation Equity Act for the 21st Century (1998) 
TRB Transportation Research Board 
TSA Transportation Security Administration 
U.S.DOT United States Department of Transportation 
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